- (Sutton, 1990) ⇒ Richard S. Sutton. (1990). “Integrated Architecture for Learning, Planning, and Reacting based on Approximating Dynamic Programming.” In: [[Proceedings of the seventh international conference (1990) on Machine learning]]. ISBN:1-55860-141-4
Subject Headings: Reinforcement Learning.
This paper extends previous work with Dyna, a class of architectures for intelligent systems based on approximating dynamic programming methods. Dyna architectures integrate trial-and-error (reinforcement) learning and execution-time planning into a single process operating alternately on the world and on a learned model of the world. In this paper, I present and show results for two Dyna architectures. The Dyna-PI architecture is based on dynamic programming's policy iteration method and can be related to existing AI ideas such as evaluation functions and universal plans (reactive systems). Using a navigation task, results are shown for a simple Dyna-PI system that simultaneously learns by trial and error, learns a world model, and plans optimal routes using the evolving world model. The Dyna-Q architecture is based on Watkins's Q-learning, a new kind of reinforcement learning. Dyna-Q uses a less familiar set of data structures than does Dyna-PI, but is arguably simpler to implement and use. We how that Dyna-Q architectures are easy to adapt for use in changing.
|1990 IntegratedArchitectureforLearni||Richard S. Sutton||Integrated Architecture for Learning, Planning, and Reacting based on Approximating Dynamic Programming||1990|