Motivated Reinforcement Learning for Improved Head Actuation of Humanoid Robots


Paper Details

  • Title: Motivated Reinforcement Learning for Improved Head Actuation of Humanoid Robots
  • Authors:
    • Jake Fountain
    • Josiah Walker
    • David Budden
    • Alexandre Mendes
    • Stephan Chalup
  • Keywords:
    • Motivated Reinforcement Learning
    • Fourier Basis
    • Head Actuation
    • Localisation
    • Simulated curiosity


The ability of an autonomous agent to self-localise within its local environment is critically dependent on its ability to make accurate observations of static, salient features. This notion has driven considerable research into the development and improvement of feature extraction and object recognition algorithms, both within RoboCup and the robotics community at large. Instead, this paper focuses on a rarely-considered issue; determining an optimal policy for actuating a robot’s head, to ensure it observes regions of the environment that will maximise the positional nformation provided. The complexity of this task is magnified by a number of common computational issues; specifically high dimensionality state spaces and noisy environmental observations. This paper details the application of motivated reinforcement learning to partially overcome these issues, leading to an 11% improvement (relative to the null case of uniformly distributed actuation policies) in self-localisation for an agent trained online for less than 1 hour. The method is demonstrated as a viable method for improving self-localisation in robotics, without the need for further optimisation of object recognition or tuning of probabilistic filters.

Publication Details

  • Status: Accepted
  • Date Accepted: 08/05/2013
  • Conference: RoboCup Symposium 2013 (proceedings to be published in Lecture Notes in Artificial Intelligence (LNAI), Springer Verlag)

How to Cite (BibTeX)

  title={Motivated reinforcement learning for improved head actuation of humanoid robots},
  author={Fountain, J. and Walker, J. and Budden, D. and Mendes, A. and Chalup, S.},
  booktitle={RoboCup 2013: Robot Soccer World Cup XVII},

Supplementary Materials

  • Original manuscript available here (the original publication will be available at
  • Presentation poster coming soon