This INCREDIBLE trick will speed up your data processes.
In this video we discuss the best way to save off data as files using python and pandas. When you are working with large datasets there comes a time when you need to store your data. Most people turn to CSV files because they are easy to share and universally used. But there are much better options out there! Watch as Rob Mulla, Kaggle grandmaster, discusses some alternative ways of saving data files: pickle, parquet and feather files. I run some benchmarks to show that you can save time, space and keep the important metadata about your files in the process!
Timeline
00:00 Intro
00:49 Creating our Data
02:08 CSVs
04:39 Setting dtypes for CSVs
06:15 Pickle Files
07:16 Parquet ❤️
09:07 Feather
10:31 Other Options
11:02 Benchmarking
12:19 Takeaways
12:43 Outro
Code Gist:
Follow me on twitch for live coding streams:
Other Videos:
1 view
98
21
4 weeks ago 00:14:36 1
TOP 40❗FAILS 4X4 THE CRAZIEST OFF ROAD ACCIDENTS ❌ INSANE FAILS AND WINS AMAZING VEHICLES 2024
1 month ago 00:16:17 1
The Ultimate Email Extractor in 2024 🚀
1 month ago 00:03:18 1
“Dancing on the Edge“ — War Thunder official cinematic trailer
1 month ago 00:17:26 1
Crossing From China🇨🇳 to Russia🇷🇺 I S2, EP98
1 month ago 00:52:21 1
The Nefertiti Scandal: Was Egypt’s Most Famous Bust Forged? | Full Documentary
1 month ago 00:20:50 1
Shocking Natural Disasters: Incredible Footage of Nature’s Fury
1 month ago 00:39:19 1
Historian Answers Witchcraft Questions | Tech Support | WIRED
1 month ago 00:12:42 1
Top 15 Upcoming City Building Games 2024 - 2025
1 month ago 00:00:31 1
Walking On Water #polarbear #northpole
1 month ago 00:04:09 1
(Everything I Do) I Do It For You - Bryan Adams (Boyce Avenue ft. Connie Talbot acoustic cover)
1 month ago 01:16:30 1
#1 Neurosurgeon: How to Manifest Anything You Want & Unlock the Unlimited Power of Your Mind
1 month ago 00:01:00 1
AWESOME Space Saving Furniture | Murphy Bed Ideas for Small Spaces Furniture Innovations
1 month ago 00:03:49 1
LINKIN PARK Epic Symphony Show - Somewhere I Belong
1 month ago 00:01:47 5
LINKIN PARK Epic Symphony Show - Teaser
1 month ago 00:02:43 1
She Keeps Bees - Burn (Session)
1 month ago 00:11:22 1
The Italian American pasta that drove the world crazy! Easy and delicious spaghetti meatballs recipe
1 month ago 02:12:31 1
Ludovico Einaudi FULL LIVE CONCERT - Arena di Verona, Italy (10/07/2024)