April 24, 2017
Last week, we finished transitioning all of our services and shared modules to Yarn, Facebook’s replacement for
npm. Here’s how we decided to make the move, how we did it, and what we learned along the way.
We've published a module to help you convert your own projects if you want to get started right away!
We were initially drawn to Yarn for two reasons: speed and better dependency pinning.
Over the past several months,
npm install had become very time-consuming for us, taking upward of 25 minutes in some projects. Since we needed to reinstall our dependencies before removing, adding, or upgrading a dependency to ensure the accuracy of our shrinkwrapping,
npm was draining a lot of time that we would have preferred to spend developing new features.
When benchmarked, Yarn proved to install dependencies more than 20x faster, even with a cold cache.
Results of one sample:
time npm install- 18:16.91
time yarn(cold cache) - 48.463
We had been using Uber’s shrinkwrapping tool since we agreed that
shrinkwrap wasn’t great. Unfortunately, Uber’s tool came with its own bugs and less-than-ideal workflow. When we began to consider Yarn, we were excited by dependency pinning being a first-class feature.
Before moving forward with Yarn, we needed to make sure that it satisfied some prerequisites:
npmin our commands.
npmif we discovered that Yarn was not a good fit for us. Our plan was simple: we’d start by only using Yarn in a few projects, and would go back to using
shrinkwrapif we decided that Yarn wasn’t right for us.
Here's what our workflow changes look like:
|Adding a package||
|Upgrading a package||
|Installing a package globally||
|Linking a local module||
(in directory to be linked)
(in directory using local module)
We began by using Yarn in one of our open-source modules, Erik, a node package for running client-side Jasmine tests headlessly. This allowed us to gain familiarity with Yarn before taking on any risk.
The next phase involved converting some of our services to use Yarn. We started with an internal service in order to minimize our risk, then moved on to other low-priority services. Along the way, we learned a few things:
yarn check, a command that “Verifies that versions of the package dependencies in the current project’s package.json matches that of yarn’s lock file”, would fail immediately after running
yarnfor the first time (which generates the
yarn.lockfile). We were able to resolve this issue most of the time by
yarnagain. Other times, the fix was more involved.
node_modulesdirectory contained within the module directory. Unlike
npm, Yarn tries to install all dependencies (transitive dependencies included) in your project’s root
node_modulesfolder, so some packages break as a result. Luckily, we were able to resolve this issue by upgrading our dependencies since the authors of the underlying packages had fixed the issue.
npmenvironment variables are implemented by Yarn (eg
npm_package_directories). We wound up circumventing this issue by ending our usage of the offending package (for reasons unrelated to Yarn), but some packages might require updates.
--mutex networkoption is handy for when you’re running multiple instances of
npm loginto regenerate our
We didn’t see any major issues running the initially converted services with Yarn after a couple of weeks, so we decided to move forward with updating our remaining services ad modules. To help make this process easier, we wrote a module that stripped out our
shrinkwrap configuration, generated a
yarn.lock file, ran the project’s tests as a sanity check, ran
yarn check to ensure the validity of the generated lockfile, and output a list of manual steps to be taken afterward in order to finish the update. This script helped us ensure that we were updating our projects correctly throughout the 75-project sweep.
In this last phase, we ran into a couple of issues:
phantomjs-prebuilton Codeship (our CI service), consistent with this issue. There are several proposed workarounds in that thread; we've added
node node_modules/phantomjs-prebuilt/install.jsto our Codeship configuration.
node_modules(in accordance with our configuration). Resetting Codeship’s dependency cache fixed these issues for us.
Yarn isn’t without its own bugs, but its community momentum and energy as well as its prioritization of speed and reliability excite us and have already proven valuable. Since last week, we’ve saved hours of development time that without Yarn would have been spent on
npm installs (and, frustratingly, repeated
npm installs after consecutive failures). Yarn has also enabled us to resume our tradition of having new engineers ship code to production on their first day, which had previously stalled due to lengthy and error-prone
Shoot us a message if you’re interested in joining a culture where engineering speed and autonomy are top priorities.