Wow, pnpm, You’re Really Fast

gowebing 2024-12-03

If you’re a Node.js developer, then you’re familiar with npm and Yarn. You might even have a strong opinion about using one over the other. For years, developers have been struggling with the bloat — in disk storage and build time — when working with Node.js package managers, especially npm.

NPM Install GIF with chickens

Source: Reddit

Then, along came pnpm, a package manager that handles package storage differently, saving users space and reducing build time. Here’s how pnpm describes the difference:

“When you install a package, we keep it in a global store on your machine, then we create a hard link from it instead of copying. For each version of a module, there is only ever one copy kept on disk. When using npm or yarn for example, if you have 100 packages using lodash, you will have 100 copies of lodash on disk. pnpm allows you to save gigabytes of disk space!”

It’s no surprise that pnpm is gaining traction, with more and more developers making it their package manager of choice. Along with that growing adoption rate, many developers who run their apps on Heroku (like I do) wanted to see pnpm supported.

Fortunately, pnpm is available via Corepack, which is distributed with Node.js. So, as of May 2024, pnpm is now available in Heroku!

In this post, we’ll cover what it takes to get started with pnpm on Heroku. And, we’ll also show off some of the storage and build-time benefits you get from using it.

A Quick Primer on pnpm

pnpm was created to solve the longstanding Node.js package manager issue of redundant storage and inefficiencies in dependency handling. npm and Yarn copy dependencies into each project’s node_modules. In contrast, pnpm keeps all the packages for all projects in a single global store, then creates hard links to these packages rather than copying them. What does this mean?

Let’s assume we have a Node.js project that uses lodash. Naturally, the project will have a node_modules folder, along with a subfolder called lodash, filled with files. To be exact, lodash@4.17.21 has 639 files and another subfolder called fp, with another 415 files.

That’s over a thousand files for lodash alone!

I created six Node.js projects: two with pnpm, two with npm, and two with Yarn. Each of them uses lodash. Let’s take a look at information for just one of the files in the lodash dependency folder.

     Shell 
   
   ~/six-projects$ ls -i npm-foo/node_modules/lodash/lodash.js
14754214 -rw-rw-r-- 544098 npm-foo/node_modules/lodash/lodash.js

~/six-projects$ ls -i npm-bar/node_modules/lodash/lodash.js
14757384 -rw-rw-r-- 544098 npm-bar/node_modules/lodash/lodash.js

~/six-projects$ ls -i yarn-foo/node_modules/lodash/lodash.js
14760047 -rw-r--r-- 544098 yarn-foo/node_modules/lodash/lodash.js

~/six-projects$ ls -i yarn-bar/node_modules/lodash/lodash.js
14762739 -rw-r--r-- 544098 yarn-bar/node_modules/lodash/lodash.js

~/six-projects$ ls -i pnpm-foo/node_modules/lodash/lodash.js
15922696 -rw-rw-r-- 544098 pnpm-foo/node_modules/lodash/lodash.js

~/six-projects$ ls -i pnpm-bar/node_modules/lodash/lodash.js
15922696 -rw-rw-r-- 544098 pnpm-bar/node_modules/lodash/lodash.js

The lodash.js file is a little over half a megabyte in size. We’re not seeing soft links, so at first glance, it really looks like each project has its own copy of this file. However, that’s not actually the case.

I used ls with the -i flag to display the inode of lodash.js file. You can see in the pnpm-foo and pnpm-bar projects, both files have the same inode (15922696). They’re pointing to the same file! That’s not the case for npm or Yarn.

So, if you have a dozen projects that use npm or Yarn, and those projects use lodash, then you’ll have a dozen different copies of lodash, along with copies from other dependencies in those projects which themselves use lodash. In pnpm, every project and dependency that requires this specific version of lodash points to the same, single, global copy.

The code for lodash@4.17.21 is just under 5 MB in size. Would you rather have 100 redundant copies of it on your machine, or just one global copy?

At the end of the day, dependency installation with pnpm is significantly faster, requiring less disk space and fewer resources. For developers working across multiple projects or managing dependencies on cloud platforms, pnpm offers a leaner, faster way to manage packages. This makes pnpm ideal for a streamlined deployment environment like Heroku.

Are you ready to start using it? Let’s walk through how.

Getting Started With pnpm

Here’s the version of Node.js we’re working with on our machine:

     Shell 
   
   $ node --version
v20.18.0

Enable and Use pnpm

As we mentioned above, Corepack comes with Node.js, so we simply need to use corepack to enable and use pnpm. We create a folder for our project. Then, we run these commands:

     Shell 
   
   ~/project-pnpm$ corepack enable pnpm

~/project-pnpm$ corepack use pnpm@latest

Installing pnpm@9.12.2 in the project...
Already up to date
Done in 494ms

This generates a package.json file that looks like this:

     JSON 
   
   {
"packageManager": "pnpm@9.12.2+sha512.22721b3a11f81661ae1ec68ce1a7b879425a1ca5b991c975b074ac220b187ce56c708fe5db69f4c962c989452eee76c82877f4ee80f474cebd61ee13461b6228"
}

This also generates a pnpm-lock.yaml file.

Next, we add dependencies to our project. For demonstration purposes, we’re copying the list of dependencies and devDependencies found in this benchmarking package.json file on GitHub. Now, our package.json file looks like this:

     JSON 
   
 
 
   {
  "version": "0.0.1",
  "dependencies": {
    "animate.less": "^2.2.0",
    "autoprefixer": "^10.4.17",
    "babel-core": "^6.26.3",
    "babel-eslint": "^10.1.0",
    ...
    "webpack-split-by-path": "^2.0.0",
    "whatwg-fetch": "^3.6.20"
  },
  "devDependencies": {
    "nan-as": "^1.6.1"
  },
  "packageManager": "pnpm@9.12.2+sha512.22721b3a11f81661ae1ec68ce1a7b879425a1ca5b991c975b074ac220b187ce56c708fe5db69f4c962c989452eee76c82877f4ee80f474cebd61ee13461b6228"
} 
  

Then, we install the packages.

     Shell 
   
   ~/project-pnpm$ pnpm install

Comparing Common Commands

The usage for pnpm is fairly similar to npm or yarn, and so it should be intuitive. Below is a table that compares the different usages for common commands (taken from this post).

npm	yarn	pnpm
`npm`	`yarn`	`pnpm`
`npm init`	`yarn init`	`corepack use pnpm@latest`
`npm install`	`yarn`	`pnpm install`
`npm install [pkg]`	`yarn add [pkg]`	`pnpm add [pkg]`
`npm uninstall [pkg]`	`yarn remove [pkg]`	`pnpm remove [pkg]`
`npm update`	`yarn upgrade`	`pnpm update`
`npm list`	`yarn list`	`pnpm list`
`npm run [scriptName]`	`yarn [scriptName]`	`pnpm [scriptName]`
`npx [command]`	`yarn dlx [command]`	`pnpm dlx [command]`
`npm exec [commandName]`	`yarn exec [commandName]`	`pnpm exec [commandName]`
`npm init [initializer]`	`yarn create [initializer]`	`pnpm create [initializer]`

Heroku Build Speed Comparison

Now that we’ve shown how to get a project up and running with pnpm (it’s pretty simple, right?), we wanted to compare the build times for different package managers when running on Heroku. We set up three projects with identical dependencies — using npm, Yarn, and pnpm.

First, we log in to the Heroku CLI (heroku login).

Then, we create an app for a project. We’ll show the steps for the npm project.

     Shell 
   
   ~/project-npm$ heroku apps:create --stack heroku-24 npm-timing

Creating ⬢ npm-timing... done, stack is heroku-24
https://npm-timing-5d4e30a1c656.herokuapp.com/ | https://git.heroku.com/npm-timing.git

We found a buildpack that adds timestamps to the build steps in the Heroku log, so that we can calculate the actual build times for our projects. We want to add that buildpack to our project, and have it run before the standard buildpack for Node.js. We do that with the following two commands:

     Shell 
   
 
 
   ~/project-npm$ heroku buildpacks:add \
                 --index=1 \
                 https://github.com/edmorley/heroku-buildpack-timestamps.git \
                 --app pnpm-timing

~/project-npm$ heroku buildpacks:add \
                 --index=2 heroku/nodejs \
                 --app npm-timing

Buildpack added. Next release on npm-timing will use:
1. https://github.com/edmorley/heroku-buildpack-timestamps.git
2. heroku/nodejs
Run git push heroku main to create a new release using these buildpacks. 
  

That’s it! Then, we push up the code for our npm-managed project.

     Shell 
   
 
 
   ~/project-npm$ git push heroku main

...
remote: Updated 4 paths from 5af8e67
remote: Compressing source files... done.
remote: Building source:
remote:
remote: -----> Building on the Heroku-24 stack
remote: -----> Using buildpacks:
remote: 1. https://github.com/edmorley/heroku-buildpack-timestamps.git
remote: 2. heroku/nodejs
remote: -----> Timestamp app detected
remote: -----> Node.js app detected
...
remote: 2024-10-22 22:31:29 -----> Installing dependencies
remote: 2024-10-22 22:31:29 Installing node modules
remote: 2024-10-22 22:31:41
remote: 2024-10-22 22:31:41 added 1435 packages, and audited 1436 packages in 11s
remote: 2024-10-22 22:31:41
remote: 2024-10-22 22:31:41 184 packages are looking for funding
remote: 2024-10-22 22:31:41 run `npm fund` for details
remote: 2024-10-22 22:31:41
remote: 2024-10-22 22:31:41 96 vulnerabilities (1 low, 38 moderate, 21 high, 36 critical)
remote: 2024-10-22 22:31:41
remote: 2024-10-22 22:31:41 To address issues that do not require attention, run:
remote: 2024-10-22 22:31:41 npm audit fix
remote: 2024-10-22 22:31:41
remote: 2024-10-22 22:31:41 To address all issues possible (including breaking changes), run:
remote: 2024-10-22 22:31:41 npm audit fix --force
remote: 2024-10-22 22:31:41
remote: 2024-10-22 22:31:41 Some issues need review, and may require choosing
remote: 2024-10-22 22:31:41 a different dependency.
remote: 2024-10-22 22:31:41
remote: 2024-10-22 22:31:41 Run `npm audit` for details.
remote: 2024-10-22 22:31:41 npm notice
remote: 2024-10-22 22:31:41 npm notice New minor version of npm available! 10.8.2 -> 10.9.0
remote: 2024-10-22 22:31:41 npm notice Changelog: https://github.com/npm/cli/releases/tag/v10.9.0
remote: 2024-10-22 22:31:41 npm notice To update run: npm install -g npm@10.9.0
remote: 2024-10-22 22:31:41 npm notice
remote: 2024-10-22 22:31:41
remote: 2024-10-22 22:31:41 -----> Build
remote: 2024-10-22 22:31:41
remote: 2024-10-22 22:31:41 -----> Caching build
remote: 2024-10-22 22:31:41 - npm cache
remote: 2024-10-22 22:31:41
remote: 2024-10-22 22:31:41 -----> Pruning devDependencies
remote: 2024-10-22 22:31:44
remote: 2024-10-22 22:31:44 up to date, audited 1435 packages in 4s
remote: 2024-10-22 22:31:44
remote: 2024-10-22 22:31:44 184 packages are looking for funding
remote: 2024-10-22 22:31:44 run `npm fund` for details
remote: 2024-10-22 22:31:45
remote: 2024-10-22 22:31:45 96 vulnerabilities (1 low, 38 moderate, 21 high, 36 critical)
remote: 2024-10-22 22:31:45
remote: 2024-10-22 22:31:45 To address issues that do not require attention, run:
remote: 2024-10-22 22:31:45 npm audit fix
remote: 2024-10-22 22:31:45
remote: 2024-10-22 22:31:45 To address all issues possible (including breaking changes), run:
remote: 2024-10-22 22:31:45 npm audit fix --force
remote: 2024-10-22 22:31:45
remote: 2024-10-22 22:31:45 Some issues need review, and may require choosing
remote: 2024-10-22 22:31:45 a different dependency.
remote: 2024-10-22 22:31:45
remote: 2024-10-22 22:31:45 Run `npm audit` for details.
remote: 2024-10-22 22:31:45 npm notice
remote: 2024-10-22 22:31:45 npm notice New minor version of npm available! 10.8.2 -> 10.9.0
remote: 2024-10-22 22:31:45 npm notice Changelog: https://github.com/npm/cli/releases/tag/v10.9.0
remote: 2024-10-22 22:31:45 npm notice To update run: npm install -g npm@10.9.0
remote: 2024-10-22 22:31:45 npm notice
remote: 2024-10-22 22:31:45
remote: 2024-10-22 22:31:45 -----> Build succeeded!
... 
  

We looked at the timing for the following steps, up until the Build succeeded message near the end:

Installing dependencies
Build
Pruning devDependencies
Caching build

In total, with npm, this build took 16 seconds.

We ran the same setup for the pnpm-managed project, also using the timings buildpack.

     Shell 
   
 
 
   ~/project-pnpm$ heroku apps:create --stack heroku-24 pnpm-timing

~/project-pnpm$ heroku buildpacks:add \
                  --index=1 \
                  https://github.com/edmorley/heroku-buildpack-timestamps.git \
                  --app pnpm-timing

~/project-pnpm$ heroku buildpacks:add \
                  --index=2 heroku/nodejs \
                  --app pnpm-timing

~/project-pnpm$ git push heroku main

…
remote: 2024-10-22 22:38:34 -----> Installing dependencies
remote: 2024-10-22 22:38:34        Running 'pnpm install' with pnpm-lock.yaml
…
remote: 2024-10-22 22:38:49        
remote: 2024-10-22 22:38:49        dependencies:
remote: 2024-10-22 22:38:49        + animate.less 2.2.0
remote: 2024-10-22 22:38:49        + autoprefixer 10.4.20
remote: 2024-10-22 22:38:49        + babel-core 6.26.3
…
remote: 2024-10-22 22:38:51 -----> Build succeeded!
 
  

For the same build with pnpm, it took only 7 seconds.

The time savings, we found, isn’t just for that initial installation. Subsequent builds, which use the dependency cache, are also faster with pnpm.

	npm	yarn	pnpm
First build	16 seconds	28 seconds	7 seconds
Second build (using cache)	15 seconds	12 seconds	7 seconds

Conclusion

When I first started Node.js development, I used npm. Several years ago, I switched to Yarn, and that’s what I had been using. . . until recently. Now, I’ve made the switch to pnpm. On my local machine, I’m able to free up substantial disk space. Builds are faster too. And now, with Heroku support for pnpm, this closes the loop so that I can use it exclusively from local development all the way to deployment in the cloud.

Happy coding!