 |
    |
Hidden Markov Modeling of Team-Play Synchronization
Itsuki Noda1,2
1Cyber Assist Reseach Center, National Institute of Advanced Industrial Science and Technology
2PRESTO, Japan Science and Technology Corporation (JST)
Abstract. Imitation Learning is considered both as a method to acquire complex human and agent behaviors, and as a way to provide seeds for further learning. However, it is not clear what is a building block in imitation learning and what is the interface of blocks; therefore, it is difficult to apply imitation learning in a constructive way. This paper addresses agents’ intentions as the building block that abstracts local situations of the agent and proposes a hierarchical hidden Markov model (HMM) in order to tackle this issue. The key of the proposed model is introduction of gate probabilities that restrict transition among agents’ intentions according to others’ intentions. Using these probabilities, the framework can control transitions flexibly among basic behaviors in a cooperative behavior. A learning method for the framework can be derived based on Baum-Welch’s algorithm, which enables learning by observation of mentors’ demonstration. Imitation learning by the proposed method can generalize behaviors from even one demonstration, because the mentors’ behaviors are expressed as a distributed representation of a flow of likelihood in HMM.
LNAI 3020, p. 102 ff.
lncs@springer.de
© Springer-Verlag Berlin Heidelberg 2004