Project Builder#

The pw_build Python module contains a light-weight build command execution library used for projects that require running multiple commands to perform a build. For example: running cmake alongside gn.

Get started

Reference

Get Started#

The quickest way to get started with Project Builder is to create a build script on existing examples.

Example Build Scripts#

Examples of Project Builder based pw build commands:

Reference#

At a high level:

A single BuildRecipe contains many BuildCommands which are run sequentially.
Multiple BuildRecipes can be created and passed into the ProjectBuilder class which provides logging and terminal output options.
A ProjectBuilder instance is then passed to the run_builds function which allows specifying the number of parallel workers (the number of recipes which are executed in parallel).

flowchart TB subgraph BuildRecipeA ["BuildRecipe: ninja"] buildCommandA1["BuildCommand gn gen out"] buildCommandA2["BuildCommand ninja -C out default"] buildCommandA1-->buildCommandA2 end subgraph BuildRecipeB ["BuildRecipe: bazel"] buildCommandB1["BuildCommand bazel build //...:all"] buildCommandB2["BuildCommand bazel test //...:all"] buildCommandB1-->buildCommandB2 end ProjectBuilder["ProjectBuilder(build_recipes=...)"] BuildRecipeA-->ProjectBuilder BuildRecipeB-->ProjectBuilder run_builds["run_builds(project_builder=..., workers=1)"] ProjectBuilder-->run_builds

class pw_build.build_recipe.BuildCommand(build_dir: ~pathlib.Path | None = None, build_system_command: str | None = None, build_system_extra_args: list[str] = <factory>, targets: list[str] = <factory>, command: list[str] = <factory>, run_if: ~typing.Callable[[~pathlib.Path], bool] = <function BuildCommand.<lambda>>)#

Store details of a single build step.

Example usage:

from pw_build.build_recipe import BuildCommand, BuildRecipe

def should_gen_gn(out: Path):
    return not (out / 'build.ninja').is_file()

cmd1 = BuildCommand(build_dir='out',
                    command=['gn', 'gen', '{build_dir}'],
                    run_if=should_gen_gn)

cmd2 = BuildCommand(build_dir='out',
                    build_system_command='ninja',
                    build_system_extra_args=['-k', '0'],
                    targets=['default']),

Parameters:

build_dir – Output directory for this build command. This can be omitted if the BuildCommand is included in the steps of a BuildRecipe.
build_system_command – This command should end with ninja, make, or bazel.
build_system_extra_args – A list of extra arguments passed to the build_system_command. If running bazel test include test as an extra arg here.
targets – Optional list of targets to build in the build_dir.
command – List of strings to run as a command. These are passed to subprocess.run(). Any instances of the '{build_dir}' string literal will be replaced at run time with the out directory.
run_if – A callable function to run before executing this BuildCommand. The callable takes one Path arg for the build_dir. If the callable returns true this command is executed. All BuildCommands are run by default.

class pw_build.build_recipe.BuildRecipe( build_dir: ~pathlib.Path, steps: list[pw_build.build_recipe.BuildCommand] = <factory>, title: str | None = None, enabled: bool = True, auto_create_build_dir: bool = True, )#

Dataclass to store a list of BuildCommands.

Example usage:

from pw_build.build_recipe import BuildCommand, BuildRecipe

def should_gen_gn(out: Path) -> bool:
    return not (out / 'build.ninja').is_file()

recipe = BuildRecipe(
    build_dir='out',
    title='Vanilla Ninja Build',
    steps=[
        BuildCommand(command=['gn', 'gen', '{build_dir}'],
                     run_if=should_gen_gn),
        BuildCommand(build_system_command='ninja',
                     build_system_extra_args=['-k', '0'],
                     targets=['default']),
    ],
)

Parameters:

build_dir – Output directory for this BuildRecipe. On init this out dir is set for all included steps.
steps – List of BuildCommands to run.
title – Custom title. The build_dir is used if this is ommited.
auto_create_build_dir – Auto create the build directory and all necessary parent directories before running any build commands.

class pw_build.project_builder.ProjectBuilder(build_recipes: ~typing.Sequence[~pw_build.build_recipe.BuildRecipe], jobs: int | None = None, banners: bool = True, keep_going: bool = False, abort_callback: ~typing.Callable = <function _exit>, execute_command: ~typing.Callable[[list, dict, ~pw_build.build_recipe.BuildRecipe, ~logging.Logger, ~typing.Callable | None], bool] = <function execute_command_no_logging>, charset: ~pw_build.project_builder.ProjectBuilderCharset = ProjectBuilderCharset(slug_ok='OK ', slug_fail='FAIL', slug_building='... '), colors: bool = True, separate_build_file_logging: bool = False, send_recipe_logs_to_root: bool = False, root_logger: ~logging.Logger = <Logger pw_build (WARNING)>, root_logfile: ~pathlib.Path | None = None, log_level: int = 20, allow_progress_bars: bool = True, use_verbatim_error_log_formatting: bool = False, log_build_steps: bool = False)#

Pigweed Project Builder

Controls how build recipes are executed and logged.

Example usage:

import logging
from pathlib import Path

from pw_build.build_recipe import BuildCommand, BuildRecipe
from pw_build.project_builder import ProjectBuilder

def should_gen_gn(out: Path) -> bool:
    return not (out / 'build.ninja').is_file()

recipe = BuildRecipe(
    build_dir='out',
    title='Vanilla Ninja Build',
    steps=[
        BuildCommand(command=['gn', 'gen', '{build_dir}'],
                     run_if=should_gen_gn),
        BuildCommand(build_system_command='ninja',
                     build_system_extra_args=['-k', '0'],
                     targets=['default']),
    ],
)

project_builder = ProjectBuilder(
    build_recipes=[recipe1, ...]
    banners=True,
    log_level=logging.INFO
    separate_build_file_logging=True,
    root_logger=logging.getLogger(),
    root_logfile=Path('build_log.txt'),
)

Parameters:

build_recipes – List of build recipes.
jobs – The number of jobs bazel, make, and ninja should use by passing -j to each.
keep_going – If True keep going flags are passed to bazel and ninja with the -k option.
banners – Print the project banner at the start of each build.
allow_progress_bars – If False progress bar output will be disabled.
log_build_steps – If True all build step lines will be logged to the screen and logfiles. Default: False.
colors – Print ANSI colors to stdout and logfiles
log_level – Optional log_level, defaults to logging.INFO.
root_logfile – Optional root logfile.
separate_build_file_logging – If True separate logfiles will be created per build recipe. The location of each file depends on if a root_logfile is provided. If a root logfile is used each build recipe logfile will be created in the same location. If no root_logfile is specified the per build log files are placed in each build dir as log.txt
send_recipe_logs_to_root – If True will send all build recipie output to the root logger. This only makes sense to use if the builds are run in serial.
use_verbatim_error_log_formatting – Use a blank log format when printing errors from sub builds to the root logger.

pw_build.project_builder.run_builds(project_builder: ProjectBuilder, workers: int = 1) → int#

Execute all build recipe steps.

Parameters:

project_builder – A ProjectBuilder instance
workers – The number of build recipes that should be run in parallel. Defaults to 1 or no parallel execution.

Returns:

1 for a failed build, 0 for success.