I'm using tweetnacl to generate sha512 hashes of strings and file. For strings it works quite well but i have no idea how to do it with files.
The signature of the function ist
extern "C" int crypto_hash(u8 *out, const u8 *m, u64 n);
unsigned char h;
crypto_hash(h, (unsigned char *)s.c_str(), s.size());
I'd suggest you to use a different implementation, one which you can incrementally feed in chunks.
This one for example. As the licence is bsd and the code is C with no dependencies, you can copy/paste only the 3 functions that you need without bringing a whole lib (no matter how small) into your project.
The life-cycle goes like:
repeatedly read blocks from file and feed them into
sha256_update(&ctx, buff, buffLen)
when EOF, get your digest using