0111: Make Bazel Pigweed’s Primary Build System#
Status: Open for Comments Intent Approved Last Call Accepted Rejected
Proposal Date: 2023-09-26
CL: pwrev/171695
Author: Ted Pudlik
Facilitator: Armando Montanez
Summary#
This SEED proposes that Pigweed transition to using Bazel as its primary build system, replacing GN in that role.
Pigweed is and will continue to be a multi-build-system project. As modular middleware, Pigweed aspires to be easy to integrate with existing embedded projects whatever their build system. To facilitate this, we provide BUILD files for multiple systems (Bazel, CMake, GN, Soong), as well as other distributables where applicable (npm packages, Python wheels, etc).
But Pigweed is more than just a collection of modules. Pigweed offers a quick, ergonomic way to start a new embedded project, as well as developer tooling that lets it scale from a prototype to production deployment. And if you’re starting a new project using Pigweed from day one, you will ask: which build system should I use? This is what we mean by Pigweed’s primary build system.
Pigweed’s primary build system has been GN, but soon will be Bazel.
Motivation#
GN has been Pigweed’s primary build system since inception, and we’ve developed an extensive GN build that was used to successfully ship products at scale. GN is fast and extensible, and its flexible toolchain abstraction is well-suited to the multi-target builds that arise in most embedded projects.
But GN has limitations:
Small community. GN is a niche build system: the only major open-source projects that use it are Chromium and Fuchsia. It is not championed by any major nonprofit or corporation. Few users or open-source contributors come to Pigweed with any past experience with GN.
No reusable rulesets. Pigweed has written and maintains all its GN rules: for C/C++, for Python, for Go (though those are deprecated). With Rust entering Pigweed, we are now developing GN rules for Rust. There are no built-in or community-provided rules we could adopt instead. Developing all rules in-house gives us flexibility, but requires large up-front and ongoing investments.
No hermetic builds. GN offers no sandboxing and relies on timestamps to decide if outputs need to be rebuilt. This has undesirable consequences:
The boundary between the environment produced by pw_env_setup and GN is blurred, making GN-built Pigweed as a whole hostile to systems like Docker or remote execution services.
Incremental builds can become corrupted. Deleting the output directory and environment is an undesirable but necessary piece of every Pigweed developer’s toolkit.
Reliably running only affected tests in CQ is not possible.
We would like Pigweed to recommend a build system that does not suffer from these limitations.
These limitations are not new. What’s changed is the build system landscape. When Pigweed was started years ago, GN was the best choice for a project emphasizing multi-target builds. But the alternatives have now matured.
Proposal#
The proposal is to make Bazel the recommended build system to use with Pigweed, and the best overall build system for embedded developers. This will involve a combination of contributions to Pigweed itself, to existing open-source Bazel rules we wish to reuse, and when necessary to core Bazel.
The vision is not merely to achieve feature parity with Pigweed’s GN offering while addressing the limitations identified above, but to fully utilize the capabilities provided by Bazel to produce the best possible developer experience. For example, Bazel offers native support for external dependency management and remote build execution. We will make it easy for Pigweed projects to leverage features like these.
What about GN? Pigweed’s GN support will continue, focusing on maintenance rather than new build features. No earlier than 2026, if no Pigweed projects are using GN, we may remove GN support. The approval of this SEED does not imply approval of removing GN support. This decision is explicitly deferred until a future date.
What about CMake? Because of its wide adoption in the C++ community, CMake will be supported indefinitely at the current level.
Roadmap#
This section lists the high-level milestones for Pigweed’s Bazel support, and then dives into the specific work needed to reach them.
This roadmap is our plan of record as of the time of writing, but like all SEED content it represents a snapshot in time. We are not as committed to the specific dates as we are to the general direction.
There’s no specific action that users must take by any date. But our recommendations about build system choice (embodied in docs and in what we tell people when they ask us) will change at some point.
Milestones#
M0: Good for Most. We can recommend Bazel as the build system for most new projects. We may not have full parity with GN yet, but we’re close enough that the benefits of adopting Bazel exceed the costs, even in the short run. The target date for this milestone is the end of 2023.
Out of scope for M0: Windows support. We have to start somewhere, and we’re starting with Linux and MacOS.
M1: Good for All. We can recommend Bazel for all new Pigweed projects, including ones that need Windows support. The target date is end of Q1 2024. After this date, we don’t expect any new projects to use GN.
M2: Best. We develop compelling features for embedded within the Bazel ecosystem. This will happen throughout 2024.
Technical tracks#
There are three main technical tracks:
Configurable toolchains exist for host and embedded, for C++ and Rust. A separate upcoming SEED will cover this area in detail, but the high-level goal is to make it straightforward to create families of related toolchains for embedded targets. This is required for milestone M0, except for Windows support, which is part of M1. The overall tracking issue is b/300458513.
Core build patterns (facades, multi-platform build, third-party crate deps for Rust) are established, documented, and usable.
M0:
Module configuration is supported in Bazel, b/234872811.
Bazel proto codegen is feature-complete, b/301328390.
Multiplatform build is ergonomic thanks to the adoption of platform_data and platform-based flags, b/301334234.
Clang sanitizers (asan, msan, tsan) are easy to enable in the Bazel build, b/301487567.
M1:
On-device testing pattern for Bazel projects developed and documented, b/301332139.
Sphinx documentation can be built with Bazel.
OSS Fuzz integration through Bazel.
Bootstrap for Bazel projects is excellent. This includes offering interfaces to Pigweed developer tooling like pw_console, pw_cli, etc.
M0: GN-free bootstrap for Bazel-based projects is designed and prototyped, b/274658181.
M1: Pigweed is straightforward to manage as a Bazel dependency, b/301336229.
Onboarding for users new to Pigweed-on-Bazel is easy thanks to excellent documentation, including examples.
M0:
There is a Bazel example project for Pigweed, b/299994234.
We have a “build system support matrix” that compares the features available in the three main build systems (Bazel, CMake, GN), b/301481759.
M1:
The sample project has Bazel support, b/302150820.
Alternatives#
The main alternatives to investing in Bazel are championing GN or switching to a different build system.
Champion GN#
Pigweed does not have the resources to bring GN to parity with modern build systems like Bazel, Buck2, or Meson. This is an area where we should partner with another large project rather than build capabilities ourselves.
CMake#
CMake is the most popular build system for C++ projects, by a significant margin. We already offer some CMake support in Pigweed. But it’s not a viable candidate for Pigweed’s primary build system:
No multi-toolchain builds Unlike Bazel and GN, CMake does not support multi-toolchain builds.
No Python or Rust support Again unlike Bazel and GN, CMake is primarily focused on building C++ code. But Pigweed is a multilingual project, and Python and Rust need first-class treatment.
No hermetic builds Unlike Bazel, CMake does not support sandboxing.
Many developers are attracted to CMake by its IDE support. Fortunately, IDE support for Bazel is also well-developed.
Other build systems#
There are other multi-lingual, correctness-emphasizing build systems out there, most prominently Meson and Buck2. We did not consider them realistic targets for migration at this time. They offer similar features to Bazel, and we have an existing Bazel build that’s in use by some projects, as well as a closer relationship with the Bazel community.
Open questions#
Additional SEEDs related to Bazel support are anticipated but have not yet been written. They will be linked from here once they exist.
SEED-0113: Modular Bazel C/C++ toolchain API
SEED-????: Pigweed CI/CQ for Bazel
Appendix: Why Bazel is great#
This SEED has not focused on why Bazel is a great build system. This is because we are not choosing Bazel over other major build systems, like Meson or Buck2, for its specific features. We are motivated to recommend a new build system because of GN’s limitations, and we choose Bazel because we have a pre-existing community of Bazel users, developers with Bazel experience, and a close relationship with the Bazel core team.
But actually, Bazel is great! Here are some things we like best about it:
Correct incremental builds. It’s great to be able to trust the build system to just do the right thing, including on a rebuild.
External dependency management. Bazel can manage external dependencies for you, including lazily downloading them only when needed. By leveraging this, we expect to speed up Pigweed bootstrap from several minutes to several seconds.
Remote build execution Bazel has excellent native support for executing build actions in a distributed manner on workers in the cloud. Although embedded builds are typically small, build latency and infra test latency is a recurring concern among Pigweed users, and leveraging remote builds should allow us to dramatically improve performance in this area.
Python environment management. The Python rules for Bazel take care of standing up a Python interpreter with a project-specific virtual environment, a functionality we had to develop in-house for our GN build.
Multilingual support. Bazel comes with official or widely adopted third-party rules for C++, Python, Java, Go, Rust, and other langauges.
Active community. The Bazel Slack is always helpful, and GitHub issues tend to receive swift attention.