Bio

Hi there! I am a Research Scientist at Google DeepMind. I am researching novel Deep Reinforcement Learning algorithms and contributing to large-scale efforts including Bard, Gemma and Gemini.

Prior to that, I did my PhD at Google Brain and Inria Lille (Scool team, ex-SequeL). I worked on Deep Reinforcement Learning, with a focus on credit assignment and interpretability. My advisors were Olivier Pietquin and Philippe Preux. I also collaborated with Matthieu Geist.

My PhD thesis is available for consultation (manuscript, slides).

Prior to my PhD, I got an engineering degree in Computer Science and Applied Mathematics from Télécom Paris, and an M.Sc. in Machine Learning from École Polytechnique. I then worked for two years as a Research Engineer at DreamQuark.

Outside of work, I make generative art (no GANs involved for now!). I also love music, roguelikes, high-intensity interval training and spicy food.

My CV is available online, and all my publications can be consulted here.

Publications

Factually Consistent Summarization via Reinforcement Learning with Textual Entailment Feedback
Johan Ferret*, Paul Roit*, Lior Shani*, Roee Aharoni, Geoffrey Cideron, Robert Dadashi, Matthieu Geist, Sertan Girgin, Léonard Hussenot, Orgad Keller, Nikola Momchev, Sabela Ramos, Piotr Stanczyk, Nino Vieillard, Olivier Bachem, Gal Elidan, Avinatan Hassidim, Olivier Pietquin, Idan Szpektor
ACL 2023
[ paper | code soon! ]

On Actions that Matter: Credit Assignment and Interpretability in Reinforcement Learning
Johan Ferret
PhD thesis
[ manuscript | slides ]

Lazy-MDPs: Towards Interpretable Reinforcement Learning by Learning When to Act
Johan Ferret*, Alexis Jacq*, Olivier Pietquin, Matthieu Geist
AAMAS 2022
[ paper | code soon! ]

There is no Turning Back: A Self-Supervised Approach to Reversibility-Aware Reinforcement Learning
Johan Ferret*, Nathan Grinsztajn*, Olivier Pietquin, Philippe Preux, Matthieu Geist
NeurIPS 2021
[ paper | blog post | slides | code ]

Self-Imitation Advantage Learning
Johan Ferret, Olivier Pietquin, Matthieu Geist
AAMAS 2021
[ paper | slides | code ]

Adversarially Guided Actor-Critic
Johan Ferret*, Yannis Flet-Berliac*, Olivier Pietquin, Philippe Preux, Matthieu Geist
ICLR 2021
[ paper | slides | video | code ]

Self-Attentional Credit Assignment for Transfer in Reinforcement Learning
Johan Ferret, Raphaël Marinier, Matthieu Geist, Olivier Pietquin
IJCAI 2020
[ paper | slides | video ]

Preprints

Gemma: Open Models Based on Gemini Research and Technology
Gemma Team
Technical report
[ paper ]

Direct Language Model Alignment from Online AI Feedback
Shangmin Guo, Biao Zhang, Tianlin Liu, Tianqi Liu, Misha Khalman, Felipe Llinares, Alexandre Ramé, Thomas Mesnard, Yao Zhao, Bilal Piot, Johan Ferret, Mathieu Blondel
arxiv preprint
[ paper ]

WARM: On the Benefits of Weight Averaged Reward Models
Alexandre Ramé, Nino Vieillard, Léonard Hussenot, Robert Dadashi, Geoffrey Cideron, Olivier Bachem, Johan Ferret
arxiv preprint
[ paper ]

Gemini: A Family of Highly Capable Multimodal Models
Gemini Team
Technical report
[ paper ]

A Survey of Temporal Credit Assignment in Deep Reinforcement Learning
Eduardo Pignatelli, Johan Ferret, Matthieu Geist, Hado van Hasselt, Laura Toni
arxiv preprint
[ paper ]

RLAIF: Scaling Reinforcement Learning from Human Feedback with AI Feedback
Harrison Lee, Samrat Phatale, Hassan Mansoor, Thomas Mesnard, Johan Ferret, Kellie Lu, Colton Bishop, Ethan Hall, Victor Carbune, Abhinav Rastogi, Sushant Prakash
arxiv preprint
[ paper ]

Acme: A Research Framework for Distributed Reinforcement Learning
Matthew Hoffman, Bobak Shahriari, John Aslanides, Gabriel Barth-Maron, Nikola Momchev, Danila Sinopalnikov, Piotr Stańczyk, Sabela Ramos, Anton Raichuk, Damien Vincent, Léonard Hussenot, Robert Dadashi, Gabriel Dulac-Arnold, Manu Orsini, Alexis Jacq, Johan Ferret, Nino Vieillard, Seyed Kamyar Seyed Ghasemipour, Sertan Girgin, Olivier Pietquin, Feryal Behbahani, Tamara Norman, Abbas Abdolmaleki, Albin Cassirer, Fan Yang, Kate Baumli, Sarah Henderson, Abe Friesen, Ruba Haroun, Alex Novikov, Sergio Gómez Colmenarejo, Serkan Cabi, Caglar Gulcehre, Tom Le Paine, Srivatsan Srinivasan, Andrew Cowie, Ziyu Wang, Bilal Piot, Nando de Freitas
arxiv preprint
[ paper | colab | code ]

Workshops

More Efficient Exploration with Symbolic Priors on Action Sequence Equivalence
Nathan Grinsztajn, Toby Johnstone, Johan Ferret, Philippe Preux
Deep Reinforcement Learning workshop, NeurIPS 2022
[ paper ]

Offline Credit Assignment in Deep Reinforcement Learning with Hindsight Discriminator Networks
Johan Ferret, Olivier Pietquin, Matthieu Geist
EWRL 2022
[ paper ]

Credit Assignment as a Proxy for Transfer in Reinforcement Learning
Johan Ferret, Raphaël Marinier, Matthieu Geist, Olivier Pietquin
Learning Transferable Skills workshop, NeurIPS 2019 (oral)
[ paper ]