Recently, I became frustrated with the slow speed of my JavaScript build. After running some experiments, I discovered that the bottleneck was transpiling ES2015 code to ES5 code via Babel.
To illustrate this, consider an example project consisting of:
node_modules/, containingbabel-cli@6.2.0andbabel-preset-es2015@6.1.18index.js, containingconsole.log('hello');
What happens if we transpile this 1-line script?
$ time babel --presets es2015 index.js
'use strict';
console.log('hello');
real 0m3.425s
user 0m3.194s
sys 0m0.272s
Three and a half seconds is a long time to do nothing. In fact, in both this fake project and in my real projects, transpiling was taking well over an order of magnitude longer than bundling and minifying. Why is the transpiling step so slow?
Babel has a handy debug mode that prints out each parsing step and the associated time. Maybe Babel is spinning on some parsing step or something?
$ DEBUG=babel babel --presets es2015 index.js
babel [BABEL] index.js: Parse start +0ms
babel [BABEL] index.js: Parse stop +7ms
babel [BABEL] index.js: Start set AST +2ms
babel program.body[0] ExpressionStatement: enter +3ms
babel program.body[0] ExpressionStatement: Recursing into... +0ms
babel program.body[0].expression CallExpression: enter +1ms
... (snip) ...
babel program.directives[1].directives[0] Directive: Recursing into... +0ms
babel program.directives[1].directives[0] Directive: exit +0ms
babel [BABEL] index.js: End transform traverse +0ms
babel [BABEL] index.js: Generation start +0ms
babel [BABEL] index.js: Generation end +4ms
Nope, Babel is Babelifying reasonably fast. However, I also noticed that the three second delay was occurring between typing the command, and seeing the the first line of debug output appear. This is what we refer to as, “being hit with the clue bat.”
So I turned to dtrace and started looking at file access activity, which was an eye-opening experience. Instead of going into the gory details, I’ll illustrate the problem more succinctly by counting the files under ‘node_modules’:
find node_modules/ -type f | wc
42929 42929 6736163
If Node has to open some appreciable fraction of these 43K files at startup… well, I ain’t no fancy Full Stack Engineer or nothin’, but that seems like a lot of file I/O.
Now for me at least, Babel is an indispensable development tool. I would give up minification before Babel. I would give up source maps before Babel. I might even give up vim before Babel. So how to make Babel faster?
One way would be to open fewer files.
I had been using trusty old npm 2, but npm 3 has a rewritten dependency management system, which is designed to produce an “as flat as possible” dependency tree. Which potentially means less file duplication.
So let’s throw away the npm 2 tree and install an npm 3 tree:
$ npm --version
2.14.3
$ sudo npm install -g npm
Password:
/opt/local/bin/npm -> /opt/local/lib/node_modules/npm/bin/npm-cli.js
npm@3.5.0 /opt/local/lib/node_modules/npm
$ rm -rf node_modules/
$ npm install babel-cli babel-preset-es2015
... (snip) ...
$ find node_modules/ -type f | wc
4495 4495 225968
And now for the moment of truth:
$ time babel --presets es2015 index.js
'use strict';
console.log('hello');
real 0m0.683s
user 0m0.621s
sys 0m0.079s
Switching to npm 3 yields a 6x speedup. In my real projects, the total speedup for the entire build (including bundling, minification, and source map generation) is more like 3x.
Lessons learned: Computers are pretty fast. Opening enormous numbers of files, not so much.