Frank Adler Frank Adler - 1 month ago 22
Javascript Question

Grunt and UglifyJS in Windows app development - UTF8 problems

I'm developing a Windows 8/8.1 JavaScript app. It basicallly is a port of our webapp. For deployment, I'm using grunt, grunt-contrib-uglify and grunt-contrib-cssmin (next to some more, but those are not connected to my problem.

UglifyJS is used to package all my JS files and libs in a minified bundle for each. For the CSS it's the same using CSSmin.

As the Windows app certification forces a UTF-8 encoding with signature (BOM), all my source files are encoded that way. I also did set the grunt default encoding to UTF-8 with preserveBOM to true. Though, my minified files ar encoded in us_ascii, some even in binary stream.

That behavior is the same for Windows 8.1 32bit and Mac OS Mavericks.

What can I do?

Re-saving this packed version with UTF-8 BOM encoding leads to a passed certification check, but the app is not running (white screen, looks like the minified JS is not working anymore, but it does flawlessly before re-saving).

This is what I'm doing with the JS files after concat/minify:

var buf = grunt.file.read(fileName, { encoding: null });
var missingBOM = (buf[0] !== 0xEF && buf[1] !== 0xBE && buf[2] !== 0xBB);
if (missingBOM) {
grunt.file.write(fileName, '\ufeff' + buf, { encoding: 'utf-8' });
}


Update:
I discovered that there are multiple BOMs in my merged files so I refined my script to strip those out before writing the new one:

var buf = grunt.file.read(dist + fileName, { encoding: null });
var BOM = new Buffer([0xEF,0xBB,0xBF]);

// remove multi BOMs from Buffer
var bufString = buf.toString('utf-8');
bufString = bufString.replace(BOM.toString('utf-8'), null);
buf = new Buffer(bufString, 'utf-8');

// add new UTF-8 BOM to the beginning of the file buffer
var bomFile = Buffer.concat([BOM,buf]);
grunt.file.write(dist + fileName, bomFile, { encoding: 'utf-8' });


Still, no luck -.-

Answer

The Problem really seems to be the combination of uglify merged files and BOM. My current setup minifies each JS file but doesn't merge them anymore (there's no need to, as the Windows app will store all scripts locally so requests are not a problem). I'm adding the BOM to all of them at the end using the snippet from update 2 of my main post.

My app now successfully certifies and runs flawlessly. Phew...