10 years of NLP history explained in 50 concepts | From Word2Vec, RNNs to GPT
From RNNs to Transformers to GPT-4, the leap in intelligence in Deep Learning research for Language Modelling and NLP has been a steady and educational growth. In this video, I explain 50 concepts that cover the basics of NLP like Tokenization and Word Embeddings, to seminal work like RNNs, Seq2Seq, Attention, to innovative Transformer models like BERT, GPT, XL-Net, and InstructGPT. I present the challenges we have faced in previous designs, and what the current architectures do to improve it, as well as highlight areas of improvement for future research.
For an overview of multimodal models that combine text with other input modalities like images, videos, audio, check out:
The video is divided broadly into 5 chapters, each adding up to 50 concepts that chronologically cover the advancements of NLP.
0:00 - Intro
0:47 - Basics of Language Modelling
1:44 - RNNs, Seq2Seq, Encoder-Decoders
6:18 - Understanding Transformers
8:26 - LLMs - BERT, GPT, XLNet, T5
12:54 - Human Alignment, ChatGPT, GPT4
16:20 - Outro
Thanks for watching!
#deeplearning #machinelearning #gpt #ai
Papers referenced:
Word2Vec:
GRUs:
LSTMS:
Seq2Seq:
Enc-Dec Attention:
Attention is All You Need:
GPT:
BERT:
Relative Position Embeddings:
Transformer-XL:
T5:
XL-Net:
Performer:
Linformer:
Longfromer:
LORA:
GPT-3:
InstructGPT:
GPT-4 Technical Report:
1 view
819
226
3 days ago 00:05:16 1
OOMPH! - Soll das Liebe sein? (Official Video) | Napalm Records
3 days ago 00:03:04 1
Sandra - (I’ll Never Be) Maria Magdalena • TopPop
3 days ago 00:04:50 1
NANOWAR OF STEEL - Stormwarrior Of The Storm (Official Video with Lyrics) | Napalm Records
3 days ago 00:08:28 1
KISS - Paul Stanley Guitar Solo / Black Diamond - Rock Am Ring 2010 - Sonic Boom Over Europe Tour
3 days ago 00:02:37 1
Star Racin Feat. Yoshimatsu Matsuda (Official AMV) [Prod. 7Teen]
3 days ago 00:03:07 1
Smokie - I’ll Meet You At Midnight (BBC Basil Brush Show )
3 days ago 00:40:09 1
Introducing the Nano Cortex by NeuralDSP | Secret Weapons
3 days ago 00:09:25 1
Boss RC-505 Loop Station - Multi-Instrumental Live Looping - Reinhardt Buhr
3 days ago 00:02:59 1
charitable insurance type of life insure that donates a portion of the policy premium to a charity
3 days ago 00:43:17 1
Сколько принимать Витамин D, как и почему именно Витамин Д?
3 days ago 00:03:58 1
Falling In Reverse - “ZOMBIFIED“
3 days ago 00:47:22 1
RUSSIA vs USA || V1 Beatmaking Battle 2024 || Beatsbyjblack & DaTrackAddict vs Moskvin & Nick Brown
3 days ago 00:03:09 1
~In my Bones~ Simba x Kovu I
3 days ago 00:22:58 1
How The US Took Over The World
3 days ago 00:00:16 1
EUROPE STADIUM TOUR RESCHEDULED TO 2021!
3 days ago 00:03:41 1
Paramore: Still Into You [OFFICIAL VIDEO]
3 days ago 00:14:36 1
Sting And Shaggy: NPR Music Tiny Desk Concert
3 days ago 00:12:19 1
RUSSIA, CHINA, IRAN – Global Axis of Resistance | Part 1
3 days ago 02:05:05 1
Our Deepest Episode Yet | Trash Taste #222
3 days ago 00:28:10 1
Golden Oldies Songs of 60s with Lyrics.
3 days ago 00:17:38 1
CONTROLS ON THE BORDER W/GERMANY. KREMLIN SLAMS POLAND’S IDEA. PUTIN:“10 X MORE DRONES THIS YEAR“.
3 days ago 00:50:42 1
KANONENFIEBER - Die Urkatastrophe (FULL ALBUM STREAM)
3 days ago 00:28:21 1
How to Make a Stylish Sewing Machine Cover | August Sew-Along 2024
4 days ago 00:17:56 1
3 Необъяснимых мифа авиационной истории, задачка немецких асов на логику и главный секрет Люфтваффе.