How to Split Large CSV Files Into Smaller Ones
Basically, you don’t need anything special to do it. You just need a Linux software split
.
This is a tiny script I have written to do such a task:
#!/bin/bash
sudo split -l 10000 myfile.csv -d --additional-suffix=.csv superphp
Let me explain it a little bit.
- the -l, -d, and –additional-suffix: use
man split
to check the documentation - the
superphp
in the end: it’s the prefix and you don’t need any option to specify it. Just add it to the end of the script.
Test
So, I have used the above little script to split a file of 7G. It took me less than 5 minutes and the large file was cut into 349 smaller ones.