Semantic Scholar Open Access 2023 144 sitasi

Q-Transformer: Scalable Offline Reinforcement Learning via Autoregressive Q-Functions

Yevgen Chebotar Q. Vuong A. Irpan Karol Hausman F. Xia +20 lainnya

Lihat Sumber DOI

Abstrak

In this work, we present a scalable reinforcement learning method for training multi-task policies from large offline datasets that can leverage both human demonstrations and autonomously collected data. Our method uses a Transformer to provide a scalable representation for Q-functions trained via offline temporal difference backups. We therefore refer to the method as Q-Transformer. By discretizing each action dimension and representing the Q-value of each action dimension as separate tokens, we can apply effective high-capacity sequence modeling techniques for Q-learning. We present several design decisions that enable good performance with offline RL training, and show that Q-Transformer outperforms prior offline RL algorithms and imitation learning techniques on a large diverse real-world robotic manipulation task suite. The project's website and videos can be found at https://qtransformer.github.io

Topik & Kata Kunci

Computer Science

Penulis (25)

Yevgen Chebotar

Q. Vuong

A. Irpan

Karol Hausman

F. Xia

Yao Lu

Aviral Kumar

Tianhe Yu

Alexander Herzog

Karl Pertsch

K. Gopalakrishnan

Julian Ibarz

Ofir Nachum

S. Sontakke

Grecia Salazar

Huong Tran

Jodilyn Peralta

Clayton Tan

D. Manjunath

Jaspiar Singht

Brianna Zitkovich

Tomas Jackson

Kanishka Rao

Chelsea Finn

S. Levine

Format Sitasi

APA MLA BibTeX

Chebotar, Y., Vuong, Q., Irpan, A., Hausman, K., Xia, F., Lu, Y. et al. (2023). Q-Transformer: Scalable Offline Reinforcement Learning via Autoregressive Q-Functions. https://doi.org/10.48550/arXiv.2309.10150

Akses Cepat

Lihat di Sumber doi.org/10.48550/arXiv.2309.10150

Informasi Jurnal

Tahun Terbit: 2023
Bahasa: en
Total Sitasi: 144×
Sumber Database: Semantic Scholar
DOI: 10.48550/arXiv.2309.10150
Akses: Open Access ✓