How Package Managers Actually Work: Resolution, Lock Files, and SBOM Calculation

An engineer runs npm install on Monday morning on a clean checkout. They get a working app. On Friday, after a peer's PR landed, they run the same npm install and get a different working app — same package.json, same Node version, different tree. Or worse, their CI job runs npm ci and fails with EUSAGE: Invalid: lock file's [email protected] does not satisfy left-pad@^1.4.0, because someone committed the result of npm install --legacy-peer-deps and the lock file no longer matches the manifest.

The mental model "the package manager reads my dependencies and downloads them" is doing a lot of unexamined work. It hides three separate decisions, made at different times, by different code paths, with different rules for when each one wins.

This article is for the engineering leader or senior engineer who has been shipping code for a decade, can read a package-lock.json, but gets ambushed by --legacy-peer-deps, silent lock regenerations, and SBOM tools that scan the same repository and produce two different answers about what is in it. We will walk through the model that all package managers share, the matrix of how twelve of them implement it, and the specific commands in npm, Yarn, pnpm, pip, Cargo, and Go that mutate lock files when you did not intend to. Then SBOM calculation, and why two scanners disagree.

The mental model nobody teaches

Every package manager — even the ones that pretend not to — has three layers.

The manifest is what the developer wrote. package.json, pyproject.toml, Cargo.toml, go.mod, pom.xml, Gemfile, composer.json. It expresses intent: I want express at any version compatible with ^4.18.0, I want serde at 1, I want org.springframework:spring-core:5.0.5.RELEASE. The manifest is a constraint, not a decision.

The resolver is the algorithm that turns the manifest into a concrete graph of exact versions. It reads the manifest, walks every transitive dependency, applies the resolution algorithm specific to that ecosystem (nested-tree-then-deduplicate for npm, backtracking for pip and Cargo, Minimum Version Selection for Go, nearest-wins for Maven), and emits a directed graph: your-app v1.0.0 → express v4.18.2 → body-parser v1.20.0 → qs v6.11.0.

The lock is the resolver's decision, written to disk, intended to be replayed. package-lock.json, yarn.lock, pnpm-lock.yaml, Cargo.lock, poetry.lock, Gemfile.lock, composer.lock, go.sum (paired with go.mod), gradle.lockfile. The lock is the contract between today's resolver and tomorrow's install.

Here is the reversal that catches people. A lock file isn't a record of what you installed; it's a contract for what the next install must produce, given the same manifest. Different tools obey that contract differently, and the difference is where supply-chain pain lives.

The matrix

Verify against primary sources. Every cell in this table is sourced from the official documentation linked at the bottom of the article.

Two patterns to call out. First, the languages with the longest history of dependency-hell — JavaScript (left-pad, March 2016), Python (the pre-pip-tools years), Java (Maven Central is a flat namespace built in 2002 with no lock file because the assumption was that artifacts were immutable forever) — are the ones whose package managers either added lock files late or still do not have one in the standard tool. Cargo and Bundler shipped with lock files from day one. Cargo's design memo cites Bundler as the reference. That decision saved Rust an entire decade of unproductive arguments.

Second, "default install can mutate lock" is the silent failure mode. On npm, Yarn Berry, pnpm, Cargo, uv, and Bundler, the default install will rewrite the lock file the moment the manifest and the lock disagree, without complaint. The hermetic-install command — npm ci, yarn install --immutable, pnpm install --frozen-lockfile, cargo build --locked, bundle install --frozen — is the one that fails loudly instead. If your CI runs the default install command, your CI is not enforcing the lock. Poetry and Composer are the safer counter-examples: when their manifest and lock disagree, poetry install warns and uses the locked versions, and composer install errors out — an explicit poetry lock or composer update is required to advance the lock.

What npm install actually does

npm is the right manager to anchor on, both because it has the largest install base and because its default behaviour is the most foot-gun-laden of the major managers. Walk through what actually happens.

When you type npm install in a directory with both package.json and package-lock.json, npm does roughly the following. It reads package.json to get your declared direct dependencies. It reads package-lock.json to get the previously-resolved tree, including every transitive dependency, the exact registry URL each one came from, and a Subresource Integrity hash (SHA-512 by default, SHA-1 for older entries) for each tarball. It walks the lock to confirm that the locked tree still satisfies the manifest's constraints — so if package.json says ^4.18.0 and the lock has 4.18.2, that is fine; if the manifest now says ^5.0.0, the lock is stale and npm will resolve a new tree. It downloads any missing tarballs, verifies the SRI hash on each, and writes them into node_modules.

If the lock was stale, npm rewrites package-lock.json and node_modules in place, then continues. There is no warning. No prompt. No exit code. The Friday-afternoon tree is now committed-pending in your working directory and the next git diff will show you a thousand-line lockfile churn.

The lock file is the source of truth for npm ci. For npm install, it is a hint that gets overwritten when convenient. That is the load-bearing distinction.

The commands you need to know:

npm install — reads the lock, may rewrite it. The default. Use it during development.

npm install <pkg> — adds a package to package.json (default --save-prod), updates the lock. The flags -D (save-dev), -O (save-optional), --save-peer, and -E (save-exact) all mutate both files. --no-save installs into node_modules without touching either.

npm ci — requires package-lock.json (or npm-shrinkwrap.json). Will not run without one. Wipes node_modules first, installs strictly from the lock, and never writes to package.json or the lock file. If the lock and the manifest disagree, it exits non-zero. This is the only npm command that is hermetic by construction.

npm update — bumps installed packages to the highest version satisfying the constraint in package.json, updates the lock. Will not respect pinned exact versions in the lock; the whole point of the command is to move past them.

--legacy-peer-deps — disables npm 7+'s automatic peer-dependency installation and reverts to npm 4 through 6 behavior, where peer conflicts emit warnings rather than errors. The trap: a tree resolved with --legacy-peer-deps produces a lock file that requires --legacy-peer-deps to install the same way again. The npm CI docs are explicit about this — if your lock was created with the flag, you must pass it to npm ci too. Forgetting it on the next developer's machine is how teams end up with two reproducible-but-different installs.

--strict-peer-deps — flips peer warnings to errors. The npm install docs note the inverse default: by default, conflicting peer dependencies are resolved with warnings; this flag fails the install instead.

--force — fetches remote resources even when a local copy exists. The aggressive sibling of --legacy-peer-deps; used when developers want the install to "just work" and are willing to throw away the cache and any pretense of reproducibility.

overrides — a top-level field in package.json that lets you pin a transitive dependency to a specific version, replace it with a fork, or unify versions across the tree. Critical for security-patching a transitive that the direct dep has not yet bumped. The npm documentation notes overrides only function in the root package.json — overrides declared in installed dependencies (or in workspaces) are ignored at resolution time. That asymmetry matters when SBOM tools later try to walk the tree and explain why a particular version is present.

For SBOM calculation, npm ships a native command: npm sbom. The format must be specified explicitly via --sbom-format=cyclonedx or --sbom-format=spdx — there is no default, and omitting the flag fails the command. It can also be scoped with --sbom-type (library / application / framework). With --package-lock-only it walks the lock file alone; without it, it also reads package.json files inside node_modules. The output is the resolved tree in CycloneDX or SPDX form, with PURLs (pkg:npm/[email protected]) and integrity hashes carried through from the lock.

This is where SBOMs start to disagree. npm sbom --package-lock-only and Syft scanning the same node_modules directory will produce different documents. Syft sees what is on disk. npm sees what the lock says should be on disk. After a --legacy-peer-deps install or a manual node_modules edit, those are different graphs.

Yarn Berry, pnpm, and the JavaScript war years

The JavaScript ecosystem's fight with reproducibility deserves a brief history. Until npm 5 (May 2017), there was no lock file at all. The left-pad incident (March 2016) — where an 11-line module was unpublished and broke half the public-internet build of every React project for several hours — happened in a world where every install of every package was a fresh resolution. Two months earlier, Facebook had begun the work that became Yarn classic; Yarn was first announced in October 2016 with yarn.lock as a first-class output, and Yarn 1.0 shipped in September 2017. npm 5 followed in May 2017 with package-lock.json.

Yarn Berry (v2, 2020; v3+ since 2021) is a different program from Yarn classic — same name, different resolver, different on-disk layout (Plug'n'Play instead of node_modules by default). The hermetic-install path is yarn install --immutable, which the Yarn documentation describes as "abort with an error exit code if the lockfile was to be modified." There is also --immutable-cache for the Plug'n'Play cache directory. Yarn does not have an --immutable default; if you want it in CI, you set it.

pnpm took a different path. It still uses node_modules, but every package is a symlink into a single content-addressable store at ~/.local/share/pnpm/store. pnpm-lock.yaml is the source of truth; pnpm install --frozen-lockfile is the hermetic mode and the documentation notes it defaults to true in CI environments and false locally. Yarn Berry does the same — its --immutable flag also defaults to true on CI — so among the major JavaScript managers, pnpm and Yarn Berry both flip their hermetic behavior based on CI=true. npm does not. The trade-off pnpm makes is that the node_modules symlink layout exposes the real dependency graph, where npm and Yarn classic flatten and hoist transitives in ways that let your code accidentally require() packages you never declared. That accidental-import bug — phantom dependencies — is invisible until you switch to pnpm and your build breaks. It is also invisible to most SBOM scanners.

For SBOM, pnpm and Yarn rely on third-party tools (@cyclonedx/cyclonedx-npm, cdxgen, Syft) rather than shipping a native command. The disagreement surface is wider here than with npm: each tool walks the lock file with its own assumptions about how to dereference workspace protocol entries (workspace:*), how to handle link: and portal: protocols in Yarn, and whether the pnpm symlink store counts as one component or many.

pnpm and Yarn install in two sentences

The default install reads the lock and silently rewrites it on manifest drift. The hermetic flag is --frozen-lockfile (pnpm) or --immutable (Yarn Berry), and your CI is not deterministic until one of those is in your pipeline.

Cargo: the "lock from day one" model

Cargo shipped with Cargo.lock from its first stable release in May 2015. The Rust team studied Bundler — Yehuda Katz worked on both — and decided up front that the binary case (where the lock is committed) and the library case (where it is generated fresh per consumer) would be handled by the same file with different conventions about whether to commit it.

cargo build reads Cargo.lock if present and uses its versions. If Cargo.toml has changed in a way that makes the lock insufficient, Cargo regenerates it, just like npm. The trap commands are cargo update (force-bumps to latest compatible versions and rewrites the lock) and any manifest change that adds a new dep.

The hermetic flags are --locked and --frozen. The Cargo docs are precise: --locked errors out if Cargo.lock would need to be updated; --frozen does that and prevents any network access for new dependencies. Use --locked in CI for reproducible builds; use --frozen for offline builds that should fail loudly if anything is missing.

The resolver is a backtracking algorithm that prefers higher versions and unifies on a single version where the version requirements are compatible (this is why two different crates depending on serde = "1" get the same serde instance, but two crates depending on serde = "1" and serde = "0.9" get both compiled in). The override mechanism is the [patch] section in Cargo.toml, which lets you redirect a dependency to a fork, a git ref, or a local path — and unlike npm overrides, [patch] works through transitive dependencies because the resolver re-runs against the patched graph.

For SBOM, Cargo does not ship a native command. The community tools are cargo-cyclonedx and cargo-sbom, both walk Cargo.lock. Syft can also scan a Rust project. They disagree most often on bom-ref formatting and on how to handle workspace members — whether a multi-crate workspace becomes one SBOM with one root component or one SBOM per crate.

Cargo's design has aged well. The [patch] table is the cleanest override mechanism in this article.

Go modules and the MVS controversy

Go did not have a package manager in the standard toolchain until late 2018. Before that, the community had dep, glide, godep, and several others, each with a different lock-file format. Russ Cox's proposal for what became Go modules (the "vgo" prototype, February 2018) generated significant pushback because it rejected SAT-based resolution — the approach used by npm, Cargo, and pip — in favor of an algorithm called Minimum Version Selection (MVS).

MVS is straightforward. For each module, take the maximum of all the minimum versions any other module requires. That is the version you use. If your go.mod says require example.com/foo v1.2.0 and a transitive dep says require example.com/foo v1.3.0, the build uses v1.3.0. If both said v1.2.0, the build uses v1.2.0 even if v1.4.0 is available. Versions in the build list never advance unless a go.mod somewhere in the graph asks for them. This is the opposite of npm and Cargo, which prefer the highest compatible version available at resolution time.

The controversy at the time was that MVS sounded like it would leave projects on stale, vulnerable versions. The defense — which has held up — is that MVS makes builds deterministic by default. A go build today and a go build in six months produce the same binary if go.mod has not changed, even without a lock file. There is no resolver state to capture.

Go does have go.sum, which is not a lock file in the npm sense. It is a checksum database: every module version that has ever been part of any build of this module gets a SHA-256 hash entry (h1: prefix, base64). The go command verifies every download against go.sum and refuses to proceed on mismatch. New modules get added; nothing is ever removed automatically. That is why go.sum files grow monotonically and look weird in code review.

The hermetic mode is -mod=readonly, which is the default since Go 1.16. If go.mod would need to be updated to satisfy a build, the build fails. The escape hatch is -mod=mod, which lets the build update go.mod and go.sum in place. There is also -mod=vendor, which uses the local vendor/ directory and ignores the network entirely.

For SBOM, Go ships nothing native. The standard tools are cyclonedx-gomod, syft, and spdx-sbom-generator. Because MVS is deterministic without a lock, all three should produce equivalent SBOMs from the same go.mod + go.sum — and in practice they often agree more closely than npm SBOM tools do. The disagreements that exist are usually about how to represent indirect dependencies (everything Go calls // indirect in go.mod) and whether to include the standard library.

pip, pip-tools, Poetry, and uv: Python's slow march to a lock file

Python is the cautionary tale. CPAN-style chaos in the late 1990s, easy_install in the 2000s, pip in 2008, and no standard lock file in the standard tool to this day. pip's resolver got a major upgrade in version 20.3 (November 2020), when it switched to a real backtracking resolver — before that, pip would happily install incompatible dependency trees and let your imports fail at runtime.

Plain pip install -r requirements.txt is not reproducible. The requirements.txt format is a list of constraints, not a lock. Two installs at two times can produce different trees. The opt-in security feature is --require-hashes, which makes pip refuse to install any package whose hash is not declared in requirements.txt; if you generate that file with pip-compile --generate-hashes, the result is approximately a lock file.

pip-tools (the Jazzband project) is the workaround. pip-compile reads pyproject.toml, setup.cfg, setup.py, or requirements.in and emits a fully-pinned requirements.txt with every direct and transitive dependency, optionally with hashes. pip-sync then makes the active virtualenv match the file exactly, removing packages that are no longer in it. The pip-tools docs are explicit: pip-sync is meant to be used only with requirements.txt files generated by pip-compile. If you pip install something into a synced environment, the next pip-sync will remove it.

Poetry (since 2018) brought a lock-first experience to Python, but its design choice goes the opposite way from npm. poetry.lock is a TOML file with SHA-256 hashes for every package; poetry install reads it; poetry lock regenerates it; poetry sync removes packages that are not in the lock (the older poetry install --sync is deprecated as of Poetry 2.x). Resolver is backtracking. The behaviour to know: editing pyproject.toml does not update poetry.lock. poetry install prints a warning that the lock is out of sync with the manifest and continues with the locked versions; poetry lock is the explicit regeneration command. This is closer to composer install than to npm install — and it is the right way around.

uv (Astral, 2024) is the new entrant, written in Rust and aggressively fast. uv.lock is TOML, the resolver is PubGrub (the same algorithm Dart's pub introduced and which cargo uses a variant of), and the hermetic mode is uv sync --locked. uv's pitch is the same one Yarn made in 2016: same model as the incumbent, ten to a hundred times faster.

The Python SBOM situation is the messiest in this article. There is no single source of truth. Syft, cyclonedx-py, and Poetry's own export commands can each look at the same project and produce three different SBOMs. The disagreements are real:

• A pip-installed environment with no lock file has no canonical "what is installed" — different scanners look at pip freeze, importlib.metadata, or site-packages directly, and these can drift.
• Optional dependencies marked with extras are sometimes included, sometimes not.
• Editable installs (pip install -e .) appear as a path reference in some tools and as the package name in others, with different PURLs.
• Pure-Python packages bundled inside compiled wheels (vendored libraries) are usually invisible to manifest-walking scanners and only get caught by binary-inspection tools like Syft scanning site-packages.

Python is the ecosystem where the gap between "what the lock file says" and "what is actually running in the container" is widest.

Maven, Gradle, Bundler, Composer

Briefly, because this article is already long.

Maven has no lock file. Dependency resolution uses "nearest-wins" — in the dependency tree, the version closest to the root project wins; if two are at the same depth, the first declared wins. This is documented and stable. The override mechanism is <dependencyManagement>, often centralized in a parent POM (a "BOM" — bill of materials, in the Maven sense, predating the security usage of the term). For reproducibility, teams use the maven-enforcer-plugin to fail the build if a transitive dependency tries to enter the graph without explicit approval. CycloneDX ships an official Maven plugin (cyclonedx-maven-plugin) that produces SBOMs by hooking into Maven's resolution.

Gradle has opt-in dependency locking. gradle.lockfile is a flat text file with one line per dependency in group:artifact:version format, followed by the configurations that include it. Locking is enabled per-configuration in build.gradle.kts. The Gradle docs are precise about behavior: locked versions are enforced as if declared with strictly(), so a manifest range lower than the lock gets silently upgraded to the lock, and a manifest range higher than the lock fails the build. Gradle's default conflict resolution is highest-wins, opposite of Maven's nearest-wins. CycloneDX has a Gradle plugin.

Bundler is the elder statesman — Gemfile.lock since 2010, the design Cargo borrowed. Resolver is Molinillo, a backtracking algorithm, also used by CocoaPods. bundle install does conservative updates: it re-resolves only what changed and preserves everything else. The hermetic flags are --frozen (errors on lock-file changes) and --deployment (which also requires the lock to exist, requires it to be up to date, and installs gems into vendor/bundle for isolation). Bundler's checksum support landed in 2.5 (2024) and is opt-in.

Composer has the cleanest split in this article. composer install reads composer.lock and installs exactly what is in it. composer update re-resolves and rewrites the lock. They are different commands with different intents. There is no flag that makes composer install quietly mutate the lock the way npm install does. PHP's package manager learned from npm's mistakes by being late.

Where SBOMs disagree

Three families of failure show up regularly when you generate an SBOM with two tools and diff the output.

Lock walkers vs. filesystem walkers. npm sbom --package-lock-only reads the lock file and emits the resolved tree. Syft scanning node_modules reads what is actually on disk. After a --legacy-peer-deps install, after a manual npm install --no-save, after a phantom dependency that pnpm would catch but npm hoists into the flat layout, these two answers differ. Neither is wrong; they are answering different questions. The CycloneDX spec calls this out implicitly by separating bom.metadata.lifecycles into build, pre-build, post-build, and operations — a build-time SBOM and a runtime SBOM are not the same document.

PURL and bom-ref disagreement. Two scanners can identify the same physical package but describe it differently. Cargo workspace SBOMs are a known sore point: cargo-cyclonedx and Syft format bom-ref values differently for workspace members, which causes downstream tools that join SBOMs by bom-ref to silently treat the same component as two distinct components. The CycloneDX team has been working on canonicalization guidance; check the current spec version before assuming any two tools will produce identical refs. {/* TODO: cite specific cyclonedx spec issue or release note for bom-ref canonicalization */}

Override and patch handling. An npm overrides block in the root package.json changes the resolved tree. Whether the SBOM reflects the override depends on the scanner: tools that walk the lock will show the overridden version (because npm rewrites the lock to match the override), tools that only read package.json will show the original constraint. Cargo [patch] redirects are similar — Syft and cargo-cyclonedx have historically disagreed on whether a patched dependency should appear as the patched source URL or the original crates.io PURL. {/* TODO: cite specific syft or cyclonedx-cargo issue documenting this */}

Yanked and unpublished versions. A package that was published and then yanked (PyPI), unpublished (npm, post-left-pad with restrictions), or retracted (Go, via the retract directive) may still be in your lock file. SBOM scanners that cross-reference against the live registry will flag it; scanners that only read the lock will not mention it. pip's resolver, on a fresh install, will re-resolve and may pick a different version than what is in requirements.txt if the original was yanked — a silent drift between the file and the install.

The lesson is not that one tool is right. The lesson is that the SBOM is a function of the resolver, the lock, and the scanner — and any two of those can be held constant while the third changes the answer.

What to do this week

Five concrete things, in order of effort.

1. Replace the default install command in CI with the hermetic one. npm ci, not npm install. yarn install --immutable, not yarn install. pnpm install --frozen-lockfile, not pnpm install. cargo build --locked, not cargo build. bundle install --frozen (or --deployment), not bundle install. go build with -mod=readonly (the default since 1.16, but check that nothing in your build scripts overrides it). composer install, not composer update. This is one line per repo and it does more for build reproducibility than any other change in this article.

2. Audit the scripts in your repo for commands that mutate the lock. npm install <pkg>, npm update, cargo update, poetry add, bundle update, composer update. Anywhere these run automatically — in a Makefile, a package.json script, a Dockerfile, a CI step — is somewhere your lock file is silently changing. Either move them behind a manual step or be explicit that the workflow is "regenerate the lock."

3. Generate an SBOM with two tools and diff the output. Pick npm sbom --sbom-format=cyclonedx and Syft. Pick cargo-cyclonedx and Syft. Pick cyclonedx-py and Poetry's poetry export. Then diff. Understand every difference. The tools that should agree on the same lock often do not, and the differences are where SBOM-driven workflows quietly break.

4. Verify that lock-file integrity hashes are actually checked at install time. For npm, this is automatic — SRI hashes in package-lock.json are verified on every install. For pip, it is not — you need --require-hashes and a requirements.txt generated with pip-compile --generate-hashes. For Bundler, checksums are opt-in (added in 2.5). Check each ecosystem in your stack and assume nothing.

5. Document which commands in your team's workflow are allowed to rewrite the lock. "Regenerating the lock" should be a deliberate act with a code-review trail, not a side effect of someone running npm install to test a typo fix locally. The lock file is a security artifact. Treat changes to it the way you would treat changes to a Dockerfile.

The package manager is not a passive tool that reads your dependencies and downloads them. It is a resolver, a lock writer, and a tree mutator, and every install is a chance for the second and third roles to silently override the first. The tools to keep that from happening exist. They are not the defaults.

References

• npm install — docs.npmjs.com/cli/v10/commands/npm-install
• npm ci — docs.npmjs.com/cli/v10/commands/npm-ci
• package-lock.json reference — docs.npmjs.com/cli/v10/configuring-npm/package-lock-json
• npm sbom — docs.npmjs.com/cli/v10/commands/npm-sbom
• npm overrides field — docs.npmjs.com/cli/v10/configuring-npm/package-json
• Yarn install (Berry) — yarnpkg.com/cli/install
• pnpm install — pnpm.io/cli/install
• pip dependency resolution — pip.pypa.io/en/stable/topics/dependency-resolution
• pip-tools — github.com/jazzband/pip-tools
• Poetry CLI — python-poetry.org/docs/cli
• uv — docs.astral.sh/uv
• Cargo.toml vs Cargo.lock — doc.rust-lang.org/cargo/guide/cargo-toml-vs-cargo-lock.html
• Cargo resolver — doc.rust-lang.org/cargo/reference/resolver.html
• Go modules reference — go.dev/ref/mod
• Maven dependency mechanism — maven.apache.org/guides/introduction/introduction-to-dependency-mechanism.html
• Gradle dependency locking — docs.gradle.org/current/userguide/dependency_locking.html
• Bundler bundle install — bundler.io/man/bundle-install.1.html
• Composer basic usage — getcomposer.org/doc/01-basic-usage.md