Bing Gan Bing Gan - 1 year ago 63
MySQL Question

What to do if a query result is too big

I'm using mysql connector of python3 to query from a mysql database. I have so many data that if I collect the recent 7 days' data, the size of the query will be over 10GB and therefore forced my python script to be killed. I think usually we can stream the result however I don't find a way to stream the query result in mysql conncetor. Is there any other way that I can solve the problem?

Answer Source

@Anthony Kong's comment is correct. To solve this issue, we can do fetchmany function from

After digging into the code a little bit, I found that the fetchmany function calls many fetchone to achieve "fetchmany". So I decide to use fetchone at last. Also fetchone comes with an example on the document