Ratan Kumar Ratan Kumar - 7 days ago 5
Node.js Question

Detecting the end of a piped stream operation

I am having an issue with detecting end of piped stream.
steps:


  1. Read File

  2. Creating gzip

  3. Write File



Which is the right way to detect end of piped stream read and write.

var inp = fs.createReadStream(fileName);
var out = fs.createWriteStream(fileName + '.gz');
var gzip = zlib.createGzip();
inp.pipe(gzip).pipe(out);
out.on('finish', function () { ....


vs

var inp = fs.createReadStream(fileName);
var out = fs.createWriteStream(fileName + '.gz');
var gzip = zlib.createGzip();
var zipStream = inp.pipe(gzip).pipe(out);
zipStream.on('finish', function () { ....


reference Link

Answer

As explained in the documentation https://nodejs.org/api/stream.html:

  • 'finish': is emitted after the stream.end() method has been called, and all data has been flushed to the underlying system.
  • 'end': is emitted when there is no more data to be consumed from the stream.

Events: 'finish' and 'end'

The 'finish' and 'end' events are from the stream.Writable and stream.Readable classes, respectively. The 'finish' event is emitted after stream.end() is called and all chunks have been processed by stream._transform(). The 'end' event is emitted after all data has been output, which occurs after the callback in transform._flush() has been called.

So, in your example, the order will be:

  • inp (readable stream) will emit the 'end'
  • gzip (transformer): will emit a 'end' and 'finish' (transformer is duplex stream)
  • out (writable stream): will finally emit a 'finish' when all data are written.