view raw
shriek shriek - 11 months ago 55
SQL Question

How to download, unzip and import to mysql dump all via piping?

Is it possible to unzip a gzipped file while it's downloading and feed it to mysql all in one go without having to create a physical file?

So far I've been able to unzip and feed it to mysql using the following:-

gunzip < somefile.sql.gz | pv | mysql -u myself -p somedb #pv for viewing process

The above process expects the gzipped file to be downloaded already while unzipping and feeding it to mysql.
But I haven't been able to feed to mysql while it's unzipping and downloading simultaneously.

And if this is not possible I'd like to know the why too so as to get a peace of mind.


Okay, I've figured out. Thanks to @TGray for pointing me to the right direction. The key was named pipes. Basically, you create a named pipe and start the download on that named pipes while piping that named pipe to gunzip as shown above in my post.

mkfifo mypipe
ssh cat /source/file.gz > mypipe &
gunzip < mypipe | pv | mysql -uusername -ppassword dbname