Semantic Scholar Open Access 2024 14014 sitasi

The Llama 3 Herd of Models

Abhimanyu Dubey Abhinav Jauhri Abhinav Pandey Abhishek Kadian Ahmad Al-Dahle +495 lainnya

Lihat Sumber

Abstrak

Modern artificial intelligence (AI) systems are powered by foundation models. This paper presents a new set of foundation models, called Llama 3. It is a herd of language models that natively support multilinguality, coding, reasoning, and tool usage. Our largest model is a dense Transformer with 405B parameters and a context window of up to 128K tokens. This paper presents an extensive empirical evaluation of Llama 3. We find that Llama 3 delivers comparable quality to leading language models such as GPT-4 on a plethora of tasks. We publicly release Llama 3, including pre-trained and post-trained versions of the 405B parameter language model and our Llama Guard 3 model for input and output safety. The paper also presents the results of experiments in which we integrate image, video, and speech capabilities into Llama 3 via a compositional approach. We observe this approach performs competitively with the state-of-the-art on image, video, and speech recognition tasks. The resulting models are not yet being broadly released as they are still under development.

Topik & Kata Kunci

Computer Science

Penulis (500)

Abhimanyu Dubey

Abhinav Jauhri

Abhinav Pandey

Abhishek Kadian

Ahmad Al-Dahle

Aiesha Letman

Akhil Mathur

A. Schelten

Amy Yang

Angela Fan

Anirudh Goyal

Anthony S. Hartshorn

Aobo Yang

Archi Mitra

A. Sravankumar

A. Korenev

Arthur Hinsvark

Arun Rao

Aston Zhang

Aur'elien Rodriguez

Austen Gregerson

Ava Spataru

Baptiste Rozière

B. Biron

Binh Tang

Bobbie Chern

Charlotte Caucheteux

Chaya Nayak

Chloe Bi

Chris Marra

Chris McConnell

Christian Keller

C. Touret

Chunyang Wu

C. Wong

Cristian Canton Ferrer

Cyrus Nikolaidis

Damien Allonsius

D. Song

D. Pintz

D. Livshits

David Esiobu

Dhruv Choudhary

Dhruv Mahajan

Diego Garcia-Olano

Diego Perino

Dieuwke Hupkes

Egor Lakomkin

Ehab A. AlBadawy

E. Lobanova

Emily Dinan

E. Smith

Filip Radenovic

Frank Zhang

Gabriel Synnaeve

Gabrielle Lee

Georgia Lewis Anderson

G. Nail

G. Mialon

Guanglong Pang

Guillem Cucurell

Hailey Nguyen

Hannah Korevaar

Hu Xu

Hugo Touvron

Iliyan Zarov

Imanol Arrieta Ibarra

Isabel M. Kloumann

Ishan Misra

Ivan Evtimov

Jade Copet

Jaewon Lee

Jan Geffert

Jana Vranes

Jason Park

Jay Mahadeokar

Jeet Shah

J. V. D. Linde

J. Billock

Jenny Hong

Jenya Lee

J. Fu

Jianfeng Chi

Jianyu Huang

Jiawen Liu

Jie Wang

Jiecao Yu

Joanna Bitton

Joe Spisak

Jongsoo Park

Joseph Rocca

J. Johnstun

Joshua Saxe

Ju-Qing Jia

Kalyan Vasuden Alwala

K. Upasani

Kate Plawiak

Keqian Li

Kenneth Heafield

Kevin R. Stone

Khalid El-Arini

Krithika Iyer

Kshitiz Malik

Kuen-ley Chiu

Kunal Bhalla

Lauren Rantala-Yeary

L. Maaten

Lawrence Chen

Liang Tan

Liz Jenkins

Louis Martin

Lovish Madaan

Lubo Malo

Lukas Blecher

Lukas Landzaat

Luke de Oliveira

M. Muzzi

Ma-hesh Pasupuleti

Mannat Singh

Manohar Paluri

Marcin Kardas

Mathew Oldham

Mathieu Rita

Maya Pavlova

M. Kambadur

Mike Lewis

Min Si

Mitesh Kumar Singh

Mona Hassan

Naman Goyal

Narjes Torabi

Niko-lay Bashlykov

Nikolay Bogoychev

Niladri S. Chatterji

Olivier Duchenne

Onur cCelebi

Patrick Alrassy

Pengchuan Zhang

Pengwei Li

Petar Vasić

Peter Weng

Prajjwal Bhargava

P. Dubal

Praveen Krishnan

Punit Singh Koura

Puxin Xu

Q. He

Qingxiao Dong

Ragavan Srinivasan

R. Ganapathy

Ramon Calderer

Ricardo Silveira Cabral

Robert Stojnic

R. Raileanu

Rohit Girdhar

Rohit Patel

Romain Sauvestre

Ron-nie Polidoro

Roshan Sumbaly

Ross Taylor

Ruan Silva

Rui Hou

Rui Wang

S. Hosseini

Sa-hana Chennabasappa

Sanjay Singh

Sean Bell

S. Kim

Sergey Edunov

Shaoliang Nie

Sharan Narang

S. Raparthy

Sheng Shen

Shengye Wan

Shruti Bhosale

Shun Zhang

Simon Vandenhende

Soumya Batra

Spencer Whitman

Sten Sootla

S. Collot

Suchin Gururangan

S. Borodinsky

Tamar Herman

T. Fowler

Tarek Sheasha

Thomas Georgiou

Thomas Scialom

Tobias Speckbacher

Todor Mihaylov

Tong Xiao

Ujjwal Karn

Vedanuj Goswami

Vibhor Gupta

Vignesh Ramanathan

Viktor Kerkez

Vincent Gonguet

Virginie Do

Vish Vogeti

Vladan Petrovic

Weiwei Chu

Wenhan Xiong

Wenyin Fu

Whit-ney Meers

X. Martinet

Xiaodong Wang

X. Tan

Xinfeng Xie

Xuchao Jia

Xuewei Wang

Yaelle Goldschlag

Yashesh Gaur

Yasmine Babaei

Yiqian Wen

Yiwen Song

Yuchen Zhang

Yue Li

Yuning Mao

Zacharie Delpierre Coudert

Zhengxu Yan

Zhengxing Chen

Zoe Papakipos

Aaditya K. Singh

Aaron Grattafiori

Abha Jain

A. Kelsey

Adam Shajnfeld

Adi Gangidi

Adolfo Victoria

Ahuva Goldstand

A. Menon

Ajay Sharma

Alex Boesenberg

Alex Vaughan

Alexei Baevski

Allie Feinstein

Amanda Kallet

Amit Sangani

Anam Yunus

Andrei Lupu

Andres Alvarado

A. Caples

Andrew Gu

Andrew Ho

Andrew Poulton

Andrew Ryan

Ankit Ramchandani

Annie Franco

Aparajita Saraf

Arkabandhu Chowdhury

Ashley Gabriel

Ashwin Bharambe

Assaf Eisenman

Azadeh Yazdan

Beau James

Ben Maurer

B. Leonhardi

Po-Yao (Bernie) Huang

Beth Loyd

Beto de Paola

Bhargavi Paranjape

Bing Liu

Bo Wu

B. Ni

Braden Hancock

Bram Wasti

Brandon Spence

B. Stojkovic

Brian Gamido

Britt Montalvo

Carl Parker

Carly Burton

Catalina Mejia

Changhan Wang

Changkyu Kim

Chao Zhou

Chester Hu

Ching-Hsiang Chu

Chris Cai

Chris Tindal

Christoph Feichtenhofer

Damon Civin

D. Beaty

Daniel Kreymer

Shang-Wen Li

Danny Wyatt

David Adkins

David Xu

Davide Testuggine

Delia David

Devi Parikh

Diana Liskovich

Didem Foss

Dingkang Wang

Duc Le

Dustin Holland

Edward Dowling

Eissa Jamil

Elaine Montgomery

Eleonora Presani

Emily Hahn

Emily Wood

Erik Brinkman

E. Arcaute

Evan Dunbar

Evan Smothers

Fei Sun

F. Kreuk

Feng Tian

Firat Ozgenel

Francesco Caggioni

Francisco (Paco) Guzmán

Frank J. Kanayet

Frank Seide

Gabriela Medina Florez

Gabriella Schwarz

Gada Badeer

Georgia Swee

Gil Halpern

Govind Thattai

Grant Herman

Grigory Sizov

Guangyi Zhang

G. Lakshminarayanan

Hamid Shojanazeri

Han Zou

Hannah Wang

Han Zha

Haroun Habeeb

Harrison Rudolph

Helen Suk

Henry Aspegren

Hunter Goldman

Igor Molybog

Igor Tufanov

Irina-Elena Veliche

Itai Gat

J. Weissman

James Geboski

James Kohli

Japhet Asher

Jean-Baptiste Gaya

Jeff Marcus

Jeff Tang

J. Chan

Jenny Zhen

J. Reizenstein

J. Teboul

J. Zhong

Jian Jin

Jingyi Yang

Joe Cummings

J. Carvill

Jon Shepard

J. McPhie

J. Torres

Josh Ginsburg

Junjie Wang

Kaixing(Kai) Wu

U. KamHou

Karan Saxena

Karthik Prasad

Kartikay Khandelwal

Katayoun Zand

Kathy Matosich

K. Veeraraghavan

Kelly Michelena

Keqian Li

Kun Huang

Kunal Chawla

Kushal Lakhotia

Kyle Huang

Lailin Chen

Lakshya Garg

A. Lavender

Leandro Silva

Lee Bell

Lei Zhang

Liangpeng Guo

Licheng Yu

Liron Moshkovich

Luca Wehrstedt

Madian Khabsa

Manav Avalani

M. Bhatt

M. Tsimpoukelli

Martynas Mankus

Matan Hasson

M. Lennie

Matthias Reso

M. Groshev

M. Naumov

Maya Lathi

Meghan Keneally

M. Seltzer

Michal Valko

Michelle Restrepo

M. Patel

Mik Vyatskov

Mikayel Samvelyan

Mike Clark

M. Macey

Mike Wang

Miquel Jubert Hermoso

Mo Metanat

Mohammad Rastegari

Mun-ish Bansal

N. Santhanam

Natascha Parks

Natasha White

Navy-ata Bawa

Nayan Singhal

Nick Egebo

Nicolas Usunier

Nikolay Pavlovich Laptev

Ning Dong

Ning Zhang

Norman Cheng

Oleg Chernoguz

O. Hart

Omkar Salpekar

Ozlem Kalinli

Parkin Kent

Parth Parekh

Paul Saab

Pavan Balaji

Pe-dro Rittner

Philip Bontrager

Pierre Roux

Piotr Dollár

Polina Zvyagina

Prashant Ratanchandani

P. Yuvraj

Qian Liang

Rachad Alao

R. Rodriguez

Rafi Ayub

Raghotham Murthy

Raghu Nayani

Rahul Mitra

Raymond Li

Rebekkah Hogan

Robin Battey

Rocky Wang

R. Maheswari

Russ Howes

Ruty Rinott

Sai Jayesh Bondu

Samyak Datta

Sara Chugh

Sara Hunt

Sargun Dhillon

S. Sidorov

S. Pan

Saurabh Verma

Seiji Yamamoto

Sharadh Ramaswamy

Shaun Lindsay

Sheng Feng

Shenghao Lin

Shengxin Zha

S. Shankar

Shuqiang Zhang

Sinong Wang

Sneha Agarwal

Soji Sajuyigbe

Soumith Chintala

Stephanie Max

Stephen Chen

Steve Kehoe

Steve Satterfield

Sudarshan Govindaprasad

S. Gupta

Sung-Bae Cho

Sunny Virk

Suraj Subramanian

Sy Choudhury

Sydney Goldman

Tal Remez

Tamar Glaser

Tamara Best

Thilo Kohler

Thomas Robinson

Tianhe Li

Tianjun Zhang

Tim Matthews

Timothy Chou

Tzook Shaked

Varun Vontimitta

Victoria O Ajayi

Victoria Montanez

Vijai Mohan

Vinay Kumar

Vishal Mangla

Vlad Ionescu

V. Poenaru

Vlad T. Mihailescu

V. Ivanov

Wei Li

Wenchen Wang

Format Sitasi

APA MLA BibTeX

Dubey, A., Jauhri, A., Pandey, A., Kadian, A., Al-Dahle, A., Letman, A. et al. (2024). The Llama 3 Herd of Models. https://www.semanticscholar.org/paper/40e8af970329135ec95057d73e239dab805ad128

@article{dubey2024, author = {Abhimanyu Dubey and Abhinav Jauhri and Abhinav Pandey and Abhishek Kadian and Ahmad Al-Dahle and Aiesha Letman and Akhil Mathur and A. Schelten and Amy Yang and Angela Fan and Anirudh Goyal and Anthony S. Hartshorn and Aobo Yang and Archi Mitra and A. Sravankumar and A. Korenev and Arthur Hinsvark and Arun Rao and Aston Zhang and Aur'elien Rodriguez and Austen Gregerson and Ava Spataru and Baptiste Rozière and B. Biron and Binh Tang and Bobbie Chern and Charlotte Caucheteux and Chaya Nayak and Chloe Bi and Chris Marra and Chris McConnell and Christian Keller and C. Touret and Chunyang Wu and C. Wong and Cristian Canton Ferrer and Cyrus Nikolaidis and Damien Allonsius and D. Song and D. Pintz and D. Livshits and David Esiobu and Dhruv Choudhary and Dhruv Mahajan and Diego Garcia-Olano and Diego Perino and Dieuwke Hupkes and Egor Lakomkin and Ehab A. AlBadawy and E. Lobanova and Emily Dinan and E. Smith and Filip Radenovic and Frank Zhang and Gabriel Synnaeve and Gabrielle Lee and Georgia Lewis Anderson and G. Nail and G. Mialon and Guanglong Pang and Guillem Cucurell and Hailey Nguyen and Hannah Korevaar and Hu Xu and Hugo Touvron and Iliyan Zarov and Imanol Arrieta Ibarra and Isabel M. Kloumann and Ishan Misra and Ivan Evtimov and Jade Copet and Jaewon Lee and Jan Geffert and Jana Vranes and Jason Park and Jay Mahadeokar and Jeet Shah and J. V. D. Linde and J. Billock and Jenny Hong and Jenya Lee and J. Fu and Jianfeng Chi and Jianyu Huang and Jiawen Liu and Jie Wang and Jiecao Yu and Joanna Bitton and Joe Spisak and Jongsoo Park and Joseph Rocca and J. Johnstun and Joshua Saxe and Ju-Qing Jia and Kalyan Vasuden Alwala and K. Upasani and Kate Plawiak and Keqian Li and Kenneth Heafield and Kevin R. Stone and Khalid El-Arini and Krithika Iyer and Kshitiz Malik and Kuen-ley Chiu and Kunal Bhalla and Lauren Rantala-Yeary and L. Maaten and Lawrence Chen and Liang Tan and Liz Jenkins and Louis Martin and Lovish Madaan and Lubo Malo and Lukas Blecher and Lukas Landzaat and Luke de Oliveira and M. Muzzi and Ma-hesh Pasupuleti and Mannat Singh and Manohar Paluri and Marcin Kardas and Mathew Oldham and Mathieu Rita and Maya Pavlova and M. Kambadur and Mike Lewis and Min Si and Mitesh Kumar Singh and Mona Hassan and Naman Goyal and Narjes Torabi and Niko-lay Bashlykov and Nikolay Bogoychev and Niladri S. Chatterji and Olivier Duchenne and Onur cCelebi and Patrick Alrassy and Pengchuan Zhang and Pengwei Li and Petar Vasić and Peter Weng and Prajjwal Bhargava and P. Dubal and Praveen Krishnan and Punit Singh Koura and Puxin Xu and Q. He and Qingxiao Dong and Ragavan Srinivasan and R. Ganapathy and Ramon Calderer and Ricardo Silveira Cabral and Robert Stojnic and R. Raileanu and Rohit Girdhar and Rohit Patel and Romain Sauvestre and Ron-nie Polidoro and Roshan Sumbaly and Ross Taylor and Ruan Silva and Rui Hou and Rui Wang and S. Hosseini and Sa-hana Chennabasappa and Sanjay Singh and Sean Bell and S. Kim and Sergey Edunov and Shaoliang Nie and Sharan Narang and S. Raparthy and Sheng Shen and Shengye Wan and Shruti Bhosale and Shun Zhang and Simon Vandenhende and Soumya Batra and Spencer Whitman and Sten Sootla and S. Collot and Suchin Gururangan and S. Borodinsky and Tamar Herman and T. Fowler and Tarek Sheasha and Thomas Georgiou and Thomas Scialom and Tobias Speckbacher and Todor Mihaylov and Tong Xiao and Ujjwal Karn and Vedanuj Goswami and Vibhor Gupta and Vignesh Ramanathan and Viktor Kerkez and Vincent Gonguet and Virginie Do and Vish Vogeti and Vladan Petrovic and Weiwei Chu and Wenhan Xiong and Wenyin Fu and Whit-ney Meers and X. Martinet and Xiaodong Wang and X. Tan and Xinfeng Xie and Xuchao Jia and Xuewei Wang and Yaelle Goldschlag and Yashesh Gaur and Yasmine Babaei and Yiqian Wen and Yiwen Song and Yuchen Zhang and Yue Li and Yuning Mao and Zacharie Delpierre Coudert and Zhengxu Yan and Zhengxing Chen and Zoe Papakipos and Aaditya K. Singh and Aaron Grattafiori and Abha Jain and A. Kelsey and Adam Shajnfeld and Adi Gangidi and Adolfo Victoria and Ahuva Goldstand and A. Menon and Ajay Sharma and Alex Boesenberg and Alex Vaughan and Alexei Baevski and Allie Feinstein and Amanda Kallet and Amit Sangani and Anam Yunus and Andrei Lupu and Andres Alvarado and A. Caples and Andrew Gu and Andrew Ho and Andrew Poulton and Andrew Ryan and Ankit Ramchandani and Annie Franco and Aparajita Saraf and Arkabandhu Chowdhury and Ashley Gabriel and Ashwin Bharambe and Assaf Eisenman and Azadeh Yazdan and Beau James and Ben Maurer and B. Leonhardi and Po-Yao (Bernie) Huang and Beth Loyd and Beto de Paola and Bhargavi Paranjape and Bing Liu and Bo Wu and B. Ni and Braden Hancock and Bram Wasti and Brandon Spence and B. Stojkovic and Brian Gamido and Britt Montalvo and Carl Parker and Carly Burton and Catalina Mejia and Changhan Wang and Changkyu Kim and Chao Zhou and Chester Hu and Ching-Hsiang Chu and Chris Cai and Chris Tindal and Christoph Feichtenhofer and Damon Civin and D. Beaty and Daniel Kreymer and Shang-Wen Li and Danny Wyatt and David Adkins and David Xu and Davide Testuggine and Delia David and Devi Parikh and Diana Liskovich and Didem Foss and Dingkang Wang and Duc Le and Dustin Holland and Edward Dowling and Eissa Jamil and Elaine Montgomery and Eleonora Presani and Emily Hahn and Emily Wood and Erik Brinkman and E. Arcaute and Evan Dunbar and Evan Smothers and Fei Sun and F. Kreuk and Feng Tian and Firat Ozgenel and Francesco Caggioni and Francisco (Paco) Guzmán and Frank J. Kanayet and Frank Seide and Gabriela Medina Florez and Gabriella Schwarz and Gada Badeer and Georgia Swee and Gil Halpern and Govind Thattai and Grant Herman and Grigory Sizov and Guangyi Zhang and G. Lakshminarayanan and Hamid Shojanazeri and Han Zou and Hannah Wang and Han Zha and Haroun Habeeb and Harrison Rudolph and Helen Suk and Henry Aspegren and Hunter Goldman and Igor Molybog and Igor Tufanov and Irina-Elena Veliche and Itai Gat and J. Weissman and James Geboski and James Kohli and Japhet Asher and Jean-Baptiste Gaya and Jeff Marcus and Jeff Tang and J. Chan and Jenny Zhen and J. Reizenstein and J. Teboul and J. Zhong and Jian Jin and Jingyi Yang and Joe Cummings and J. Carvill and Jon Shepard and J. McPhie and J. Torres and Josh Ginsburg and Junjie Wang and Kaixing(Kai) Wu and U. KamHou and Karan Saxena and Karthik Prasad and Kartikay Khandelwal and Katayoun Zand and Kathy Matosich and K. Veeraraghavan and Kelly Michelena and Keqian Li and Kun Huang and Kunal Chawla and Kushal Lakhotia and Kyle Huang and Lailin Chen and Lakshya Garg and A. Lavender and Leandro Silva and Lee Bell and Lei Zhang and Liangpeng Guo and Licheng Yu and Liron Moshkovich and Luca Wehrstedt and Madian Khabsa and Manav Avalani and M. Bhatt and M. Tsimpoukelli and Martynas Mankus and Matan Hasson and M. Lennie and Matthias Reso and M. Groshev and M. Naumov and Maya Lathi and Meghan Keneally and M. Seltzer and Michal Valko and Michelle Restrepo and M. Patel and Mik Vyatskov and Mikayel Samvelyan and Mike Clark and M. Macey and Mike Wang and Miquel Jubert Hermoso and Mo Metanat and Mohammad Rastegari and Mun-ish Bansal and N. Santhanam and Natascha Parks and Natasha White and Navy-ata Bawa and Nayan Singhal and Nick Egebo and Nicolas Usunier and Nikolay Pavlovich Laptev and Ning Dong and Ning Zhang and Norman Cheng and Oleg Chernoguz and O. Hart and Omkar Salpekar and Ozlem Kalinli and Parkin Kent and Parth Parekh and Paul Saab and Pavan Balaji and Pe-dro Rittner and Philip Bontrager and Pierre Roux and Piotr Dollár and Polina Zvyagina and Prashant Ratanchandani and P. Yuvraj and Qian Liang and Rachad Alao and R. Rodriguez and Rafi Ayub and Raghotham Murthy and Raghu Nayani and Rahul Mitra and Raymond Li and Rebekkah Hogan and Robin Battey and Rocky Wang and R. Maheswari and Russ Howes and Ruty Rinott and Sai Jayesh Bondu and Samyak Datta and Sara Chugh and Sara Hunt and Sargun Dhillon and S. Sidorov and S. Pan and Saurabh Verma and Seiji Yamamoto and Sharadh Ramaswamy and Shaun Lindsay and Sheng Feng and Shenghao Lin and Shengxin Zha and S. Shankar and Shuqiang Zhang and Sinong Wang and Sneha Agarwal and Soji Sajuyigbe and Soumith Chintala and Stephanie Max and Stephen Chen and Steve Kehoe and Steve Satterfield and Sudarshan Govindaprasad and S. Gupta and Sung-Bae Cho and Sunny Virk and Suraj Subramanian and Sy Choudhury and Sydney Goldman and Tal Remez and Tamar Glaser and Tamara Best and Thilo Kohler and Thomas Robinson and Tianhe Li and Tianjun Zhang and Tim Matthews and Timothy Chou and Tzook Shaked and Varun Vontimitta and Victoria O Ajayi and Victoria Montanez and Vijai Mohan and Vinay Kumar and Vishal Mangla and Vlad Ionescu and V. Poenaru and Vlad T. Mihailescu and V. Ivanov and Wei Li and Wenchen Wang}, title = {The Llama 3 Herd of Models}, year = {2024}, journal = {Semantic Scholar}, url = {https://www.semanticscholar.org/paper/40e8af970329135ec95057d73e239dab805ad128}, }

Akses Cepat

PDF tidak tersedia langsung

Cek di sumber asli →

Lihat di Sumber

Informasi Jurnal

Tahun Terbit: 2024
Bahasa: en
Total Sitasi: 14014×
Sumber Database: Semantic Scholar
Akses: Open Access ✓