Developmental Machine Learning, Curiosity and Deep Reinforcement Learning
Speaker(s) : Pierre-Yves OUDEYER (INRIA)
Current approaches to AI and machine learning are still fundamentally limited in comparison with autonomous learning capabilities of children. What is remarkable is not that some children become world champions in certains games or specialties: it is rather their autonomy, flexibility and efficiency at learning many everyday skills under strongly limited resources of time, computation and energy. And they do not need the intervention of an engineer for each new task (e.g. they do not need someone to provide a new task specific reward function). I will present a research program (Kaplan and Oudeyer, 2004; Oudeyer et al., 2007; Gottlieb and Oudeyer, 2019) that has focused on computational modeling of child development and learning mechanisms in the last decade. I will discuss several developmental forces that guide exploration in large real world spaces, starting from the perspective of how algorithmic models can help us understand better how they work in humans, and in return how this opens new approaches to autonomous machine learning. In particular, I will discuss models of curiosity-driven autonomous learning, enabling machines to sample and explore their own goals and their own learning strategies, self-organizing a learning curriculum without any external reward or supervision. I will introduce the Intrinsically Motivated Goal Exploration Processes (IMGEPs-) algorithmic framework, and present two families of IMGEPs: population-based IMGEPs (Baranes and Oudeyer, 2013; Forestie et al.,2017) with learned goal spaces (Pere et al., 2018), which have allowed sample efficient learning learning of skill repertoires in real robots, and goal-conditioned Deep RL-based IMGEPs, which enable strong generalization properties when they are modular (Colas et al., 2019), in particular when leveraging the compositionality of language to imagine goals in curiosity-driven exploration (Colas et al., 2020).
Séminaire organisé conjointement avec l'IEEE France Section Life Members Affinity Group et le chapitre Computational Intelligence de l’IEEE France Section (http://ieee-ci.lip6.fr/).
Christophe.Marsala (at) nulllip6.fr