I asked AI to make a Music Video... the results are trippy
In this video, I utilized artificial intelligence to generate an animated music video for the song Canvas by Resonate. This tool allows anyone to generate beautiful images using only text as the input. My question was, what if I used song lyrics as input to the AI, can I make perfect music synchronized videos automatically with the push of a button? Let me know how you think the AI did in this visual interpretation of the song.
After getting caught up in the excitement around DALL·E2 (latest and greatest AI system, it’s INSANE), I searched for any way I could use similar image generation for music synchronization. Since DALL·E2 is not available to the public yet, my search led me to VQGAN CLIP (Vector Quantized Generative Adversarial Network and Contrastive Language–Image Pre-training), before settling more specifically on Disco Diffusion V5.2 Turbo. If you don’t know what any of these words or acronyms mean, don’t worry, I was just as confused when I first started learning about this technology. I believe we’re reaching a turning point where entire industries are about to shift in reaction to this new process (which is essentially magic!).
Important note:
While this AI is impressive, it still required additional input beyond just the song lyrics to achieve the music video I was looking for. For example, I added keyframes for camera motion throughout the generated world. These keyframes were manually synchronized to the beat by me. I also specified changes to the art style at different moments of the song. Since many of the lyrics are quite non-specific, even a human illustrator would have a hard time making visual representations. To make the lyrics more digestible by the AI, I sometimes modified the phrase to be more coherent, such as specifying a setting or atmosphere.
This was my first time working with DDV5, and I’m very happy with the results! There were many times where my jaw dropped upon seeing what the AI came up with. I haven’t felt this sense of wonder from technology since I first experienced a HD videogame as a child.
If you would like to learn more about how this video was made, try this yourself, or ask me any questions, I’ll post a more detailed explanation of how to get started on Patreon (link below). The post is free to the public, no need to pay. If you do want to support me and become a member that would be much appreciated, you’ll also automatically be entered into the end screen minigames where you earn points on each video and move up the leaderboard!
Join on Patreon to automatically have your name included in the next video:
Want to add lyrics and color beat blocks to your Disco Diffusion project like I did in my video? Here is my code:
My social media:
Twitter:
Discord:
Tiktok: @doodlechaos
Shorts Channel:
Instagram Shorts:
Email: contact@
While Disco Diffusion is based on the contributions of many, show some love to the two most prominent contributors:
Music:
[Indie Dance] - Rezonate - Canvas [Monstercat EP Release] :
1 view
170
31
8 months ago 00:54:52 1
#3 Люди, составляющие церковь. Проповедь Александра Шевченко.
8 months ago 00:15:51 1
Судебные страсти. Сбежавшая мать / Crazy Court. Runaway Mother
8 months ago 00:13:51 1
Судебные страсти. Горе аферист / Crazy Court. Grief Conman
8 months ago 00:13:22 1
English Lesson 1 - Hello. What’s your name? | English with cartoons and songs from Gogo
8 months ago 00:00:34 1
the post-nut interview
8 months ago 00:01:25 1
The CORRECT DIMEBAG TUNING! (D 425hz!)
8 months ago 00:04:51 1
Whitney Houston - I Have Nothing (Official HD Video)
8 months ago 00:07:37 1
Five Finger Death Punch - Judgement Day (Official Music Video)
8 months ago 00:04:45 1
Микробиом кишечника ► Как бактерии кишечника влияют на наше настроение?
8 months ago 00:04:32 1
Christina Aguilera - Hurt (Official Video)
8 months ago 00:43:39 1
Зачем Deus Ex удаляет твой прогресс [Хардмод]
8 months ago 00:37:06 1
Not here to arrest, Here to help you !
8 months ago 00:01:23 1
What if Simba died? Sad story😢❤️ Read the description please
8 months ago 00:04:02 1
When You Have Zero Awareness
8 months ago 00:03:16 1
A$AP Rocky - Goldie (Official Video)
8 months ago 00:04:34 1
Whitney Houston - I Will Always Love You (Official 4K Video)
8 months ago 00:09:44 1
My Gear on Tour (2021)
8 months ago 00:08:13 1
Rachel Zegler FIRED!
8 months ago 00:02:25 1
I Have A Pet | Animal Song | Super Simple Songs
8 months ago 00:01:30 1
Chinese propaganda song - Hitler AI Cover (Red sun in the sky)
8 months ago 00:05:05 1
Gavriil asks his siters to sing his favourite song more and more - Гавриил просит сестер петь романс
8 months ago 00:10:43 2
[Корпорация Озвучки] ASK FUFEL I Ответы на вопросы - ЧАСТЬ 1
8 months ago 00:03:25 1
Laura Cox - All Right Now - Free cover
8 months ago 00:03:40 1
Cee-Lo - I’ll Be Around (Official Video) ft. Timbaland