Accelerating Transformers via Kernel Density Estimation Insu Han
A Google TechTalk, presented by Insu Han, 2023/05/30
A Google Algorithms Seminar. ABSTRACT: Dot-product attention mechanism plays a crucial role in modern deep architectures (e.g., Transformer) for sequence modeling, however, naïve exact computation of this model incurs quadratic time and memory complexities in sequence length, hindering the training of long-sequence models. Critical bottlenecks are due to the computation of partition functions in the denominator of softmax function as well as the multiplication of the softmax matrix with the matrix of values. Our key observation is that the former can be reduced to a variant of the kernel density estimation (KDE) problem, and an efficient KDE solver can be further utilized to accelerate the latter via subsampling-based fast matrix products. Our proposed KDEformer can approximate the attention in sub-quadratic time with provable spectral norm bounds, while all prior results merely provide entry-wise error bounds. Empirically, we verify that KDEformer outperforms other attention approximations in terms of accuracy, memory, and runtime on various pre-trained models.
Bio: Insu Han is a postdoctoral research fellow at Yale University, hosted by Amin Karbasi. He completed his Ph.D. degree in the School of Electrical Engineering at the Korea Advanced Institute of Science and Technology (KAIST) in 2021, under the supervision of Jinwoo Shin. Before that, he obtained his Bachelor’s degree in Electrical Engineering and minored in Mathematics at KAIST. He has worked on developing and analyzing approximate algorithms for large-scale machine learning problems and their applications. His most recent work focuses on accelerating the attention mechanism in large language models via fast kernel density estimation methods. In 2019, he was the recipient of the Microsoft Research Asia Fellowship.
1 view
0
0
5 days ago 00:45:45 1
All 9 Solfeggio Frequencies in 45 Minute | Complete Restoration of Body & Mind | Black Screen
5 days ago 00:20:34 1
Chosen Ones, GOD Will Make You RICH AND FAMOUS Cuz You DON’T GIVE AF about BEING RICH AND FAMOUS🔥
6 days ago 02:01:47 1
IL EST TEMPS D’EXPOSER CLAIREMENT LA SUPERCLASSE MONDIALE ET APATRIDE | MICHEL GEOFFROY | GPTV
1 week ago 00:56:25 1
Au-delà de la civilisation ? Entretiens avec Michel Bauwens. Episode 2
1 week ago 00:07:36 1
AI Generates Open-World Games?
1 week ago 00:16:47 2
La Scission de la Terre S’accélère - Nous Sommes à un Point de Non-Retour
2 weeks ago 00:34:46 1
How to Improve Your Accent || Graded Reader || Learn English Through Story || Improve Your English
2 weeks ago 01:29:17 1
Film LE SECRET Film complet en français LA LOI DE L’ATTRACTION Puissant Inspirant et Motivant
2 weeks ago 00:17:28 2
17 SEPTEMBRE : UN TOURNANT DÉCISIF POUR LES 6 PROCHAINS MOIS DE MANIFESTATION DE LA PLEINE LUNE
2 weeks ago 01:17:12 1
Cartographier l’Anthropocène à l’ère de l’intelligence artificielle – Lancement de l’Atlas IGN 2024
2 weeks ago 00:11:44 1
2 Game-Changing AI Stocks to Buy Now for Long-Term Gains
2 weeks ago 00:00:20 1
30 Seconds to Tranquility: Find Your Calm Fast 🌿✨ #RelaxingSounds #PeaceInSeconds #nature #relax
2 weeks ago 00:00:14 1
Quiet Your Mind and Embrace Inner Peace 🌊✨ #RelaxingMusic #FindYourCalm #nature #relax
2 weeks ago 00:11:17 1
Changements Globaux et la Transition de la Terre vers la Nouvelle Ère
2 weeks ago 00:00:15 1
Release Tension and Find Peace in Just 30 Seconds 🌿✨ #RelaxingMusic #CalmWithin #nature #relax
2 weeks ago 00:00:24 1
Soothing Sounds: Discover Inner Peace in Just 30 Seconds 🌟🌿 #RelaxingMusic #InnerCalm #relax
2 weeks ago 00:00:27 1
Guided by Music: Find Your Calm in 30 Seconds 🌟🌿 #RelaxingMusic #PeacefulBreak #beachlife #beach
3 weeks ago 00:06:07 1
ĐIỀU TRỊ MỤN VIÊM TỤ CẦU VÙNG TRÁN, CAM KẾT HIỆU QUẢ SAU 1 BUỔI, KHÔNG THÂM, KHÔNG ĐỂ LẠI SẸO.
3 weeks ago 00:00:15 1
Reconnect with Peace: 30 Seconds of Calming Music 🌿✨ #InstantCalm #RelaxingMusic #nature #relax
3 weeks ago 00:00:16 1
Instant Calm: Reconnect with Serenity in 30 Seconds 🌿✨ #RelaxingMusic #QuickCalm #nature #relax
3 weeks ago 00:00:20 1
Instant Relaxation: Calm Your Mind in 30 Seconds 🌿✨ #RelaxNow #QuickCalm #nature #stressrelief
3 weeks ago 00:00:22 1
Embrace Peace: Release Tension and Find Your Calm 🌿💫 #RelaxingMusic #InnerPeace #relaxing #nature
3 weeks ago 00:34:45 1
Chosen Ones: Seeing THIS Means You’re on the HIGHEST Timeline shift! 🚀