Aro Aro - 20 days ago 6
Dart Question

Is it possible to dynamically produce large files (10Gb) and stream them to the client?

Is it possible to dynamically produce large files (10Gb+) for the client to download?

I'm building a webapp that dynamically creates files and downloads them to the client. I've implemented this by creating a Blob of data and using an objectUrl to download them.

(example from: Download large files in Dartlang):

import 'dart:html';

void main() {
var data = new Blob(['Hello World!\n'*1000000]);
querySelector("#downloadLink")
..setAttribute('href', Url.createObjectUrl(data));
}


However, this does not work for large files.
Is there a way to stream files to the client, or append data to a file, so that the file can be generated piece by piece rather than generating the entire thing before downloading?

Answer

for < 800MB i would recommend FileSaver but for 10GB+ you are going to need something like StreamSaver (It will only work in Blink doe) 16GB+ haven't been any problem for me

const fileStream = streamSaver.createWriteStream('filename.txt')
const writer = fileStream.getWriter()
const encoder = new TextEncoder()

// When you have any data you write it as Uint8array
let data = 'a'.repeat(1024)
let uint8array = encoder.encode(data + "\n\n")

writer.write(uint8array) // chunk
writer.write(uint8array) // chunk
writer.write(uint8array) // chunk
writer.write(uint8array) // chunk

// After you have written all bytes you need to close it
writer.close()

(just don't write all chunks at once, do it progressively as you get more data avalible)