reproducible builds

Building Secure and Reliable Systems: Best Practices for Designing, Implementing, and Maintaining Systems
by Heather Adkins , Betsy Beyer , Paul Blankinship , Ana Oprea , Piotr Lewandowski and Adam Stubblefield
Published 29 Mar 2020

Reproducible Running the same build commands on the same inputs is guaranteed to produce bit-by-bit identical outputs. Reproducibility almost always requires hermeticity.18 Reproducible builds have the following benefits: Verifiability—A verifier can determine the binary provenance of an artifact by reproducing the build themselves or by using a quorum of rebuilders, as described in “Verifiable Builds”. Hermeticity—Nonreproducibility often indicates nonhermeticity. Continuously testing for reproducibility can help detect nonhermeticity early, thereby ensuring all the benefits of hermeticity described earlier. Build caching—Reproducible builds allow for better caching of intermediate build artifacts in large build graphs, such as in Bazel.

…

Input artifacts should generally list both an identifier, such as a URI, and a version, such as a cryptographic hash. You typically use the identifier to verify the authenticity of the build—for example, to verify that code came from the proper source repository. The version is useful for various purposes, such as ad hoc analysis, ensuring reproducible builds, and verification of chained build steps where the output of step i is the input to step i+1. Be aware of the attack surface. You need to verify anything not checked by the build system (and therefore implied by the signature) or included in the sources (and therefore peer reviewed) downstream.

…

Similarly, if the user can specify a custom compiler, then the verifier must ensure that the “right” compiler was used. In general, the more validation the build process can perform, the better. For a good example of binary provenance, see Debian’s deb-buildinfo format. For more general advice, see the Reproducible Builds project’s documentation. For a standard way to sign and encode this information, consider JSON Web Tokens (JWT). Code Signing Code signing is often used as a security mechanism to increase trust in binaries. Use care when applying this technique, however, because a signature’s value lies entirely in what it represents and how well the signing key is protected.

Software Engineering at Google: Lessons Learned From Programming Over Time
by Titus Winters , Tom Manshreck and Hyrum Wright
Published 17 Mar 2020

But there are still a few problems that came up earlier that we haven’t addressed. Bazel has clever ways of solving each of these, and we should discuss them before moving on. Tools as Dependencies One problem we ran into earlier was that builds depended on the tools installed on our machine, and reproducing builds across systems could be difficult due to different tool versions or locations. The problem gets even harder when your project uses languages that require different tools based on which platform they’re being built on or compiled for (e.g. Windows vs. Linux), and each of those platforms requires a slightly different set of tools to do the same job.

The Rust Programming Language, 2nd Edition
by Steve Klabnik and Carol Nichols
Published 27 Feb 2023

When you build your project in the future, Cargo will see that the Cargo.lock file exists and will use the versions specified there rather than doing all the work of figuring out versions again. This lets you have a reproducible build automatically. In other words, your project will remain at 0.8.5 until you explicitly upgrade, thanks to the Cargo.lock file. Because the Cargo.lock file is important for reproducible builds, it’s often checked into source control with the rest of the code in your project. Updating a Crate to Get a New Version When you do want to update a crate, Cargo provides the command update, which will ignore the Cargo.lock file and figure out all the latest versions that fit your specifications in Cargo.toml.

…

If you open the src/main.rs file, make a trivial change, and then save it and build again, you’ll only see two lines of output: $ cargo build Compiling guessing_game v0.1.0 (file:///projects/guessing_game) Finished dev [unoptimized + debuginfo] target(s) in 2.53 secs These lines show that Cargo only updates the build with your tiny change to the src/main.rs file. Your dependencies haven’t changed, so Cargo knows it can reuse what it has already downloaded and compiled for those. Ensuring Reproducible Builds with the Cargo.lock File Cargo has a mechanism that ensures you can rebuild the same artifact every time you or anyone else builds your code: Cargo will use only the versions of the dependencies you specified until you indicate otherwise. For example, say that next week version 0.8.6 of the rand crate comes out, and that version contains an important bug fix, but it also contains a regression that will break your code.

The Rust Programming Language
by Steve Klabnik and Carol Nichols
Published 14 Jun 2018

If you open the src/main.rs file, make a trivial change, and then save it and build again, you’ll only see two lines of output: $ cargo build Compiling guessing_game v0.1.0 (file:///projects/guessing_game) Finished dev [unoptimized + debuginfo] target(s) in 1.50 secs These lines show Cargo only updates the build with your tiny change to the src/main.rs file. Your dependencies haven’t changed, so Cargo knows it can reuse what it has already downloaded and compiled for those. It just rebuilds your part of the code. Ensuring Reproducible Builds with the Cargo.lock File Cargo has a mechanism that ensures you can rebuild the same artifact every time you or anyone else builds your code: Cargo will use only the versions of the dependencies you specified until you indicate otherwise. For example, what happens if next week version 0.3.15 of the rand crate comes out and contains an important bug fix but also contains a regression that will break your code?

…

When you build a project for the first time, Cargo figures out all the versions of the dependencies that fit the criteria and then writes them to the Cargo.lock file. When you build your project in the future, Cargo will see that the Cargo.lock file exists and use the versions specified there rather than doing all the work of figuring out versions again. This lets you have a reproducible build automatically. In other words, your project will remain at 0.3.14 until you explicitly upgrade, thanks to the Cargo.lock file. Updating a Crate to Get a New Version When you do want to update a crate, Cargo provides another command, update, which will ignore the Cargo.lock file and figure out all the latest versions that fit your specifications in Cargo.toml.

Elixir in Action
by Saša Jurić
Published 30 Jan 2019

Working with dependencies 287 Running mix deps.get fetches all dependencies (recursively) and stores the reference to the exact version of each dependency in the mix.lock file, unless mix.lock already exists on the disk, in which case this file is consulted to fetch the proper versions of dependencies. This ensures reproducible builds across different machines, so make sure you include mix.lock in the source control where your project resides. Now that you’ve fetched all of your dependencies, you can build the entire system by running mix compile, which will compile all the dependencies and the project. It’s worth mentioning that Poolboy is an Erlang library, but mix will still know how to compile it. 11.2.2 Adapting the pool With these preparations in place, you can start adapting the pool implementation.

Site Reliability Engineering: How Google Runs Production Systems
by Betsy Beyer , Chris Jones , Jennifer Petoff and Niall Richard Murphy
Published 15 Apr 2016

Some build systems like Bazel8 have valuable features that afford more precise control over testing. For example, Bazel creates dependency graphs for software projects. When a change is made to a file, Bazel only rebuilds the part of the software that depends on that file. Such systems provide reproducible builds. Instead of running all tests at every submit, tests only run for changed code. As a result, tests execute cheaper and faster. There are a variety of tools to help you quantify and visualize the level of test coverage you need [Cra10]. Use these tools to shape the focus of your testing: approach the prospect of creating highly tested code as an engineering project rather than a philosophical mental exercise.

Programming Rust: Fast, Safe Systems Development
by Jim Blandy and Jason Orendorff
Published 21 Nov 2017

In the rare case that your project is a shared library (i.e., the output is a .dll, .dylib, or .so file), there is no such downstream cargo user, and you should therefore commit Cargo.lock. Cargo.toml’s flexible version specifiers make it easy to use Rust libraries in your project and maximize compatibility among libraries. Cargo.lock’s bookkeeping supports consistent, reproducible builds across machines. Together, they go a long way toward helping you avoid dependency hell. Publishing Crates to crates.io You’ve decided to publish your fern-simulating library as open source software. Congratulations! This part is easy. First, make sure Cargo can pack the crate for you