至于本文提到的重生强化，所提出的算法PVRL（policy (+data) to value RL），他们和上述五个方案的异同点在于，和offline RL一样，利用teacher policy的data做了离线预训练，后面的在线调优，和Kickstarting一样，都用了策略蒸馏损失，和他们不一样的在于，这篇工作对策略蒸馏损失加了一个衰减系数，作为“断奶”的操作。 一共就三步，而且最后一个不同其实也就多了一个超参数。 但人家是第一个正式定义大模型预训练范式，并且在很多任务中都验证好使，也算是一个solid …

2022年6月3日 · Equipped with this algorithm, we demonstrate reincarnating RL's gains over tabula rasa RL on Atari 2600 games, a challenging locomotion task, and the real-world problem of navigating stratospheric balloons.

2022年11月3日 · To address the inefficiencies of tabula rasa RL, we present “Reincarnating Reinforcement Learning: Reusing Prior Computation To Accelerate Progress” at NeurIPS 2022. Here, we propose an alternative approach to RL research, where prior computational work, such as learned models, policies, logged data, etc., is reused or transferred between ...

2022年10月31日 · Equipped with this algorithm, we demonstrate reincarnating RL's gains over tabula rasa RL on Atari 2600 games, a challenging locomotion task, and the real-world problem of navigating stratospheric balloons.

2022年12月14日 · 至于本文提到的重生强化，所提出的算法PVRL（policy (+data) to value RL），他们和上述五个方案的异同点在于，和offline RL一样，利用teacher policy的data做了离线预训练，后面的在线调优，和Kickstarting一样，都用了策略蒸馏损失，和他们不一样的在于，这篇工作对策略蒸馏损失加了一个衰减系数，作为“断奶”的操作。 一共就三步，而且最后一个不同其实也就多了一个超参数。 但人家是第一个正式定义大模型预训练范式，并且在很多任务中都验 …

Equipped with this algorithm, we demonstrate reincarnating RL’s gains over tabula rasa RL on Atari 2600 games, a challenging locomotion task, and the real-world problem of navigating stratospheric balloons.

Equipped with this algorithm, we demonstrate reincarnating RL's gains over tabula rasa RL on Atari 2600 games, a challenging locomotion task, and the real-world problem of navigating stratospheric balloons.

To address the inefficiencies of tabula rasa RL and help unlock the full potential of deep RL, this workshop would focus on the alternative paradigm of leveraging prior computational work, referred to as reincarnating RL, to accelerate training across design iterations of an RL agent or when moving from one agent to another.

This work argues for an alternative approach to RL research, where we build on prior computational work, which we believe could significantly improve real-world RL adoption and help democratize it further.

Learning “tabula rasa”, that is, from scratch without much previously learned knowledge, is the dominant paradigm in reinforcement learning (RL) research. However, learning tabula rasa is the exception rather than the norm for solving larger-scale problems.