alternate npm and bower client focused on security and performance.

At Facebook we use npm heavily for all of our open source JavaScript projects. Unfortunately npm is lacking in key areas that are important to us. The changes we've made are quite extensive and require significant changes to npm including the internal architecture and workflow for casual users.

fbkpm is written from scratch and uses the existing npm registry for module hosting, it's completely compatible with the npm ecosystem and is only an alternative to the npm client. With a focus on security, fbkpm helps auditing of third party dependencies by providing more insight into changes. This is done by requiring all new and updated dependencies to be manually audited with the use of custom diff tooling that makes the review of large amounts of code very easy.

While security is a priority, we also offer the following features to assist in the stability and development of frontend dependency management.

• Greatly improved performance: Client is always performing operations such as package resolving and fetching due to the internal architecture.
• Dependency deduping: Reduces the duplication of modules reducing code size.
• Ability to manually resolve version conflicts so only a single version per package exists. (Bower style)
• Resilient to network flakiness: Smart retry and timeout logic ensures reliability on poor networks such as conference wifi, mobile networks etc.
• Stable public API: Tooling such as build systems can reliably hook into internals.
• Reproducible builds: Updates are intentional by default which means builds cannot randomly break because of dependency updates.
• More emojis. 🐈

Security

• Locked dependencies by default.
• Prompts user for verification for running install scripts.
• Prompts user for verification when connecting to foreign hosts.
• Refuses to download a tarball from HTTP without a hash.
• Refuses to download a tarball from HTTPS if an HTTP redirect was included.
• Refuses to clone a git repo over HTTP and plain git.
• Blacklists known bad hosts that ping analytics servers.
• Lockfile contains tarball hashes to ensure integrity of package downloads.
• Generates reports when installing and updating dependencies. Contains diffs of module code and analyses modules for possible points of conflict such as suspicious code.

Installing react-native, 700 transitive dependencies with no cache over WiFi on a 25 megabit down connection.

$ git clone git@github.com:facebook/fbkpm.git
$ cd fbkpm
$ npm install
$ make build
$ npm link
# go into some random directory
$ mkdir node_modules
$ fbkpm install your-package

Similar in many aspects but unlike existing alternate npm clients, fbkpm is focused on security and determinism first, excellent performance is just an implementation detail. If we only cared about performance we'd help improve npm as that doesn't require a change in workflow.

The changes we've made significantly alter the workflow that developers who use npm are used to. We believe this is for the best but we understand that this isn't for everyone.

We hope that some of our ideas make their way back into the main npm client.

NOTE: This is not an exhaustive list and is not a direct comparison to npm.

The npm dependency graph is nondeteriministic and is dependent on install order. This is terrible for reproducible builds.

This is extremely important for ecosystems relying on Bower today. It makes it very hard to reason about what modules you're actually using and can lead to bloat and dependency creep.

Loading modules is traditionally very easy with a flat structure as you have a single place to look for modules making the resolution extremely fast.

A flat structure can easily be implemented through the use of the previously mentioned CAS storage. After package resolution is performed, a list of all duplicate packages are built and it's reduced to a single version via a console UI and some analysis.

Shrinkwrap See the more security section.

Deduping Due to the use of a CAS system, there doesn't need to be any deduping as modules are stored in one central folder so all duplications resolve to the same module. Due to the directory structure introduced by CAS, modules cannot access others higher in their tree meaning you only have access to the modules you've specified in your package.json

Prune extraneous It's extremely common for the following scenario to occur:

• Kim and Bob are working on a project.
• Kim and Bob have the exact same lock checkout (that includes babel as a dependency).
• Kim removes babel from package.json.
• Kim commits updated lockfile, package.json and pushes.
• Bob downloads, and runs fbkpm install.
• Bob doesn't realise that Kim removed the babel dependency and uses it in his code because it still exists in his local node_modules.

This can be mitigated by automatically pruning extraneous modules on install.

Locked dependencies by default By default npm does not enforce the use of a lockfile. This means that a lot of users aren't protected from accidental breakages.

Verification of executing lifecycle commands When installing packages from npm, any package can run arbitrary commands on package install. This is dangerous and allows for a variety of vulnerabilities. Prompting the user for verification to run an install script and storing the answer in the shrinkwrap will ensure that only verified commands are executed.

Verification of connecting to foreign hosts It's unfortunately common for large packages to track users. When connecting to foreign hosts that we don't have explicitly in a whitelist we should prompt the user for verification. Once the resolved version has been stored in the shrinkwrap, the check will be skipped.

Analysis of new packages and updated packages It's hard to know what new dependencies, owners, and potential access have been granted when installing and updating packages. There needs to be more insight into what has actually been introduced with an update. This can be achieved through tool assisted code and metadata review. Showing you explicitly what new dependencies have been introduced, the files that have been touched and alerting you to possible patterns worth investigation. It's important for any type of analysis to be extremely explicit so there's no false sense of security and laziness when reviewing new third party dependencies.

Network reliability TODO

Network performance TODO

Parallelism TODO

Lots of build systems today are consuming npm. The npm API relies a lot on global state, can exit the process mid-command if it gets grumpy and has inconsistent logging and reporting behaviour.

One project goal is to have an extremely flexible API that build tools can consume.