Martin Adámek Martin Adámek -4 years ago 83
Javascript Question

Create zip archive larger than 4 GiB that can be opened via MacOS Archive Utility

I am trying to generate large zip file (without compression, just storing the files) on the fly and stream it via

restify
. The zip file can be bigger than 4 GiB in total and I ran to issue with ZIP64 format, that seems not to be supported properly on MacOS.

I tried using
node-archiver
and
yazl
, they are both able to generate the zip file, but I am not able to open it via native graphical utility in MacOS (e.g. just double-clicking on the file in Finder). The weirdest thing about this is, that the native
unzip
command line utility works fine here, just the graphical Archive Utility is not working.

I also tried to create the zip using simple java implementation and the resulting file works ok even with the graphical Archive Utility, so the problem is probably in implementation of
node-archiver
/
yazl
libraries. I also tried to create the zip via command line (
zip file.zip -0 ...files
), and it can also be opened via Archive Utility.

What is the correct way to handle this situation? Is there some other node library that can produce ZIP64 format that can be opened via Archive Utility?

It is necessary to stream the zip file, not storing it on disk. I am running latest version of MacOS sierra (10.12.5 beta) and node 6.9.1.

Answer Source

If anyone struggles with the same requirements as I did, check JSZip library (https://stuk.github.io/jszip/). It can also work with node streams and is able to produce zip files larger than 4 GiB, that can be opened via MacOS Archive Utility. The only limitation of this is, that you can not have a file larger than 4 GiB, because of 32 bit CRC32 hash generation (but the zip file can be larger).

Here is a snippet of how you can use this library:

const fs = require('fs');
const JSZip = require('jszip');

const files = [...];
const zip = new JSZip();

for (const file of files) {
  zip.file(file.name, fs.createReadStream(file.path));
}

res.header('Content-Type', 'application/zip');
res.header('Content-Disposition', `attachment; filename="${encodeURI(`test.zip`)}"`);
zip.generateNodeStream({streamFiles: true, compression: 'STORE'}).pipe(res);
Recommended from our users: Dynamic Network Monitoring from WhatsUp Gold from IPSwitch. Free Download