Our Mad Journey of Building a Vector Database in Go - Weaviate at FOSDEM 2023

Building a database from scratch is no small feat. Doing so in Go might just be pure madness. In this talk, Co-Founder & CTO Etienne Dilocker highlights the unique challenges and their creative solutions that our development team encountered while building Weaviate from scratch in Go. Right in the beginning, we’ve made the decision to own as much as possible of the critical infrastructure within Weaviate. This meant that we initially had to build many things that might have already existed somewhere. But today, this kind of control over our architecture allows us to iterate fast and deliver innovative new solutions that make Weaviate faster, better scaling, and more reliable than anything out there. “Our Mad Journey of Building a Vector Database in Go” was recorded in February of 2023 at FOSDEM, Brussels. It is a summary of the crazy things we had to do to make Weaviate what it is today – so you don’t have to. Buckle up for a mad journey, so you can later sit back, relax and trust that Weaviate has got you covered. Chapters 1:00 Introduction 1:26 Why does the world need yet another database? 3:57 Memory Allocations 9:40 Delayed Decoding 16:05 SIMD 22:04 Demo Time! 24:38 Mad at Go? 26:00 Audience Questions