Sigmun Sigmun - 7 months ago 52
Python Question

Use numpy.load on file within zipfile

I have a zipfile wich contains many npy files (

file1.npy
,
file2.npy
,
file3.npy
,...). I would like to load them individually without extract zipfile on a filesystem. I have tried many things but I can't figure it out.

My guess was :

import zipfile
import numpy as np

a = {}

with zipfile.ZipFile('myfiles.zip') as zipper:
for p in zipper.namelist():
with zipper.read(p) as f:
a[p] = np.load(f)


Any ideas?

Answer

Does the numpy function expect a file object, not the resulting text. For zip files, I generally do something like:

with ZipFile(path, mode='r') as archive:
    with io.BufferedReader(archive.open(filename, mode='r')) as file:

I am guessing you should pass zipper.open(p, mode='r') into np.load(f). Also, I strong urge you not to do zipper.read(p) since it will read the whole file in memory at once. So, using your code conventions, try:

with zipfile.ZipFile('myfiles.zip') as zipper:
    for p in zipper.namelist():
        with io.BufferedReader(zipper.open(p, mode='r')) as f:
            a[p] = np.load(f)
Comments