UNIVERSITY OF HERTFORDSHIRE
		 COMPUTER SCIENCE RESEARCH COLLOQUIUM


          "EMBER: Reinforcement Learning in Embodied Agents"

			     David Jacob 
		       School of Computer Science
		       University of Hertfordshire

			26 October 2005 (Wednesday)

			  Lecture Theatre E350
		       Hatfield, College Lane Campus
			       3 - 4 pm

	       Coffee/tea and biscuits will be available.
			 [Catering Permitting]

		    Everyone is Welcome to Attend
			  [Space Permitting]


Abstract:

Reinforcement Learning is a seductive idea: by simply
providing an agent with rewards to indicate whether it is
doing what we want or not, it can be trained to perform any
task which can be represented as a Markov Decision Process.
However the size of state and action spaces makes learning
very slow and thus renders the approach impractical for any
realistic embodied agent.

Various methods have been proposed to make learning more
tractable. My research focusses on the decomposition of
agents into modules based on the details of the agent's
embodiment: these modules sense, act and learn
independently.

This has a number of desirable outcomes: it makes learning
practical, it allows the agent to learn a set of transferable
competences which can be brought to any task, and
it provides a simple mechanism for controlling (or `shaping')
the agent's behaviour to improve learning rates and
task performance. In addition, the small state spaces and
close coupling of action and reward enable adaptive action
sequences to be learned, making for more usable skills than
are typically achievable using open loop control. We
demonstrate this generic skill learning using a simulation
of a quadruped walker robot which learns a dynamically
stable gait and performs it in a system with billions of
states.

This works was carried out in the Adaptive Systems Research
Group as part of a PhD under the supervision of Drs. Daniel Polani, 
C. L. Nehaniv, and David Lee.
--
Hertfordshire Computer Science Research Colloquium
http://homepages.feis.herts.ac.uk/~nehaniv/colloq