Code formatting#

pw_presubmit: Python tools for running presubmit checks and linters

The pw format command provides a unified experience for checking and fixing code formatting for a variety of languages used in Pigweed. It automatically detects which formatters to apply based on file extensions and names.

Features#

Check vs. fix: Run in check-only mode with --check to find errors without modifying files.
Targeted formatting: Format specific files or directories by passing them as arguments.
Only format modified files: By default, only formats the files that have been modified in the last change (or since the upstream tracking branch, if set). Use --full to format all files in a repo.
Automatic configuration: Discovers and uses project-specific formatter configurations like .clang-format, .black.toml, and rustfmt.toml.
Parallel execution: Speed up formatting with the -j or --jobs flag to run on multiple files in parallel.
Unified experience: Regardless of what files you format, you’ll always have a consistent CLI experience.

Language support:

File type	Flag	Enabled by default
Bazel build files	`--@pigweed//pw_presubmit/py:enable_bazel_formatter`	✅
C/C++	`--@pigweed//pw_presubmit/py:enable_cpp_formatter`	✅
CMake	`--@pigweed//pw_presubmit/py:enable_cmake_formatter`	✅
CSS	`--@pigweed//pw_presubmit/py:enable_css_formatter`	✅
GN build files	`--@pigweed//pw_presubmit/py:enable_gn_formatter`	❌
Go	`--@pigweed//pw_presubmit/py:enable_go_formatter`	❌
Java	`--@pigweed//pw_presubmit/py:enable_java_formatter`	✅
JavaScript	`--@pigweed//pw_presubmit/py:enable_javascript_formatter`	❌
JSON	`--@pigweed//pw_presubmit/py:enable_json_formatter`	✅
Markdown	`--@pigweed//pw_presubmit/py:enable_markdown_formatter`	✅
OWNERS files	`--@pigweed//pw_presubmit/py:enable_owners_formatter`	✅
Protobuf	`--@pigweed//pw_presubmit/py:enable_protobuf_formatter`	✅
Python (using `black`)	`--@pigweed//pw_presubmit/py:enable_python_formatter`	✅
reStructuredText	`--@pigweed//pw_presubmit/py:enable_rst_formatter`	✅
Rust	`--@pigweed//pw_presubmit/py:enable_rust_formatter`	❌
Starlark	`--@pigweed//pw_presubmit/py:enable_starlark_formatter`	✅
TypeScript	`--@pigweed//pw_presubmit/py:enable_typescript_formatter`	❌

Get started#

The formatter is usually invoked via the pw entry point at the root of your project.

Usage examples#

# Fix formatting for all files.
$ ./pw format

# Check formatting for all files in the repository without making
# modifications.
$ ./pw format --check

# Format all files tracked by Git.
$ ./pw format --all

# Format specific files or directories.
$ ./pw format pw_foo/ public/pw_foo/foo.h

Project setup#

Pigweed’s default formatter is designed to work out-of-the-box for most Bazel projects. You can try it with no additional setup setups with the following command:

$ bazel run @pigweed//pw_presubmit/py:format

Add a shortcut#

To make running the formatter easier, add the format to a Workflow shortcut. See the Workflows Get Started guide for more info.

Enable additional formatters#

Some formatters that may not be interesting to most projects are disabled by default, and may be enabled by adding flags to your project’s .bazelrc file:

# Enable GN build file formatting.
common --@pigweed//pw_presubmit/py:enable_gn_formatter=True

Note

Enabling additional formatters will cause more tooling to be downloaded.

API reference#

Note

b/326309165: While the pw format command interface is very stable, the pw_presubmit.format library is a work-in-progress effort to detach the implementation of pw format from the pw_presubmit module. Not all formatters are migrated, and the library API is unstable. After some of the core pieces land, this library will be moved to pw_code_format.

Core#

Formatting library core.

pw_presubmit.format.core.DiffCallback

The callback type for producing diffs.

Arugments:: path: File path of the file being diffed. original: The contents of the original file, as a string. formatted: The contents of the formatted file, as a string.

Returns:: A human readable diff as a string.

alias of Callable[[Path, str, str], str]

class pw_presubmit.format.core.FileChecker(*, file_patterns: ~pw_cli.file_filter.FileFilter, mnemonic: str, tool_runner: ~pw_cli.tool_runner.ToolRunner = <pw_cli.tool_runner.BasicSubprocessRunner object>, diff_tool: ~typing.Callable[[~pathlib.Path, str, str], str] = <function simple_diff>)

Abstract class for a code format check tool.

This class does not have the ability to apply formatting to files, and instead only allows in-memory checks to produce expected resulting diffs.

file_patterns: A pw_cli.file_filter.FileFilter that describes what kind of files this check applies to.

mnemonic: A human-readable description of the kind of checker this is (e.g. “C and C++”, “Bazel”, “Python (black)”).

tool_runner: The pw_presubmit.format.core.ToolRunner to use when calling out to subprocesses.

diff_tool: The pw_presubmit.format.core.DiffCallback to use when producing formatting diffs.

__init__(*, file_patterns: ~pw_cli.file_filter.FileFilter, mnemonic: str, tool_runner: ~pw_cli.tool_runner.ToolRunner = <pw_cli.tool_runner.BasicSubprocessRunner object>, diff_tool: ~typing.Callable[[~pathlib.Path, str, str], str] = <function simple_diff>)

abstract format_file_in_memory( file_path: Path, file_contents: bytes, ) → FormattedFileContents

Returns the formatted version of a file as in-memory bytes.

file_path and file_content represent the same file. Both are provided for convenience. Use file_path if you can do so without modifying the file, or use file_contents if the formatting tool provides a mechanism for formatting the file by piping it to stdin.

Any subprocess calls should be initiated with self.run_tool() to enable testing and injection of tools and tool wrappers.

WARNING: A pw_presubmit.format.core.FileChecker must never modify the file at``file_path``.

Returns:: A populated pw_presubmit.format.core.FormattedFileContents that contains either the result of formatting the file, or an error message.

get_formatting_diff(file_path: Path, dry_run: bool = False) → FormattedDiff | None

Returns a diff comparing a file to its formatted version.

If dry_run is True, the diff will always be None.

Returns:: None if there is no difference after formatting OR if dry_run is enabled. Otherwise, a pw_presubmit.format.core.FormattedDiff is returned containing either a diff or an error.

get_formatting_diffs( paths: Iterable[Path], dry_run: bool = False, ) → Iterator[FormattedDiff]

Checks the formatting of many files without modifying them.

This method may be overridden to optimize for formatters that allow checking multiple files in a single invocation, though you may need to do additional parsing to produce diffs or error messages associated with each file path.

Returns:: An iterator of pw_presubmit.format.core.FormattingDiff objects for each file with identified formatting issues.

class pw_presubmit.format.core.FileFormatter(**kwargs)

Abstract class for a code format fix tool.

__init__(**kwargs)

abstract format_file(file_path: Path) → FormatFixStatus

Formats the provided file in-place.

Any subprocess calls should be initiated with self.run_tool() to enable testing and injection of tools and tool wrappers.

Returns:: A FormatFixStatus that contains relevant errors/warnings.

format_files( paths: Iterable[Path], keep_warnings: bool = True, ) → Iterator[tuple[Path, FormatFixStatus]]

Formats the provided files and fixes them in-place.

All files must be updated to contain the formatted version. If errors are encountered along the way, they should be collected and returned as a dictionary that maps the path of the file to a string that describes the errors encountered while processing that file.

Any subprocess calls should be initiated with self.run_tool() to enable testing and injection of tools and tool wrappers.

This method may be overridden to optimize for formatters that allow formatting multiple files in a single invocation, though you may need to do additional parsing to associate error messages with the paths of the files that produced them.

Returns:: An iterator of Path and pw_presubmit.format.core.FormatFixStatus pairs for each file that was not successfully formatted. If keep_warnings is True, any successful format operations with warnings will also be returned.

class pw_presubmit.format.core.FormatFixStatus(ok: bool, error_message: str | None)

The status of running a code formatter in-place on a file.

This type is returned by in-place formatting fix operations.

ok

A boolean indicating whether or not formatting was successful.

Type:: bool

error_message

A string containing any errors or warnings produced by the formatting process.

Type:: str | None

__init__(ok: bool, error_message: str | None) → None

class pw_presubmit.format.core.FormattedDiff(ok: bool, diff: str, error_message: str | None, file_path: Path)

The resulting diff of applying a code formatter to a file.

ok

A boolean indicating whether or not formatting was successful.

Type:: bool

diff

The resulting diff of applying code formatting, as a human-readable string.

Type:: str

error_message

A string containing any errors or warnings produced by the formatting process.

Type:: str | None

file_path

The path of the corresponding file that produced this diff.

Type:: pathlib.Path

__init__(ok: bool, diff: str, error_message: str | None, file_path: Path) → None

class pw_presubmit.format.core.FormattedFileContents(ok: bool, formatted_file_contents: bytes, error_message: str | None)

The result of running a code formatter on the contents of a file.

This type is returned by in-memory formatting check operations.

ok

A boolean indicating whether or not formatting was successful.

Type:: bool

formatted_file_contents

The contents of the resulting formatted file as bytes.

Type:: bytes

error_message

A string containing any errors or warnings produced by the formatting process.

Type:: str | None

__init__(ok: bool, formatted_file_contents: bytes, error_message: str | None) → None

pw_presubmit.format.core.simple_diff(path: Path, original: str, formatted: str) → str: Produces a diff of the contents of two files.

Formatters#

class pw_presubmit.format.bazel.BuildifierFormatter(warnings_to_fix: Sequence[str] = ('load', 'native-build', 'unsorted-dict-items'), **kwargs)

A formatter that runs buildifier on files.

__init__(warnings_to_fix: Sequence[str] = ('load', 'native-build', 'unsorted-dict-items'), **kwargs)

format_file(file_path: Path) → FormatFixStatus

Formats the provided file in-place using buildifier.

Returns:: A FormatFixStatus that contains relevant errors/warnings.

format_file_in_memory( file_path: Path, file_contents: bytes, ) → FormattedFileContents

Uses buildifier to check the formatting of the requested file.

The file at file_path is NOT modified by this check.

Returns:: A populated pw_presubmit.format.core.FormattedFileContents that contains either the result of formatting the file, or an error message.

format_files( paths: Iterable[Path], keep_warnings: bool = True, ) → Iterator[Tuple[Path, FormatFixStatus]]

Uses buildifier to format the specified files in-place.

Returns:: An iterator of Path and pw_presubmit.format.core.FormatFixStatus pairs for each file that was not successfully formatted. If keep_warnings is True, any successful format operations with warnings will also be returned.

class pw_presubmit.format.cpp.ClangFormatFormatter(tool_flags: Sequence[str] = ('--style=file',), **kwargs)

A formatter that runs clang-format on files.

__init__(tool_flags: Sequence[str] = ('--style=file',), **kwargs)

format_file(file_path: Path) → FormatFixStatus

Formats the provided file in-place using clang-format.

Returns:: A FormatFixStatus that contains relevant errors/warnings.

format_file_in_memory( file_path: Path, file_contents: bytes, ) → FormattedFileContents

Uses clang-format to check the formatting of the requested file.

The file at file_path is NOT modified by this check.

Returns:: A populated pw_presubmit.format.core.FormattedFileContents that contains either the result of formatting the file, or an error message.

format_files( paths: Iterable[Path], keep_warnings: bool = True, ) → Iterator[tuple[Path, FormatFixStatus]]

Uses clang-format to format the specified files in-place.

Returns:: An iterator of Path and pw_presubmit.format.core.FormatFixStatus pairs for each file that was not successfully formatted. If keep_warnings is True, any successful format operations with warnings will also be returned.

class pw_presubmit.format.gn.GnFormatter(tool_flags: Sequence[str] = (), **kwargs)

A formatter that runs gn format on files.

__init__(tool_flags: Sequence[str] = (), **kwargs)

format_file(file_path: Path) → FormatFixStatus

Formats the provided file in-place using gn format.

Returns:: A FormatFixStatus that contains relevant errors/warnings.

format_file_in_memory( file_path: Path, file_contents: bytes, ) → FormattedFileContents

Uses gn format to check the formatting of the requested file.

The file at file_path is NOT modified by this check.

Returns:: A populated pw_presubmit.format.core.FormattedFileContents that contains either the result of formatting the file, or an error message.

format_files( paths: Iterable[Path], keep_warnings: bool = True, ) → Iterator[Tuple[Path, FormatFixStatus]]

Uses gn format to fix formatting of the specified files in-place.

Returns:: An iterator of Path and pw_presubmit.format.core.FormatFixStatus pairs for each file that was not successfully formatted. If keep_warnings is True, any successful format operations with warnings will also be returned.

class pw_presubmit.format.python.BlackFormatter(config_file: Path | bool = True, **kwargs)

A formatter that runs black on files.

__init__(config_file: Path | bool = True, **kwargs)

Creates a formatter for Python that uses black.

Parameters:: config_file – The black config file to use to configure black. This defaults to True, which formats files using the nearest .black.toml file in the parent directory of the file being formatted. False disables this behavior entirely.

format_file(file_path: Path) → FormatFixStatus

Formats the provided file in-place using black.

Returns:: A FormatFixStatus that contains relevant errors/warnings.

format_file_in_memory( file_path: Path, file_contents: bytes, ) → FormattedFileContents

Uses black to check the formatting of the requested file.

The file at file_path is NOT modified by this check.

Returns:: A populated pw_presubmit.format.core.FormattedFileContents that contains either the result of formatting the file, or an error message.

format_files( paths: Iterable[Path], keep_warnings: bool = True, ) → Iterator[Tuple[Path, FormatFixStatus]]

Uses black to format the specified files in-place.

Returns:: An iterator of Path and pw_presubmit.format.core.FormatFixStatus pairs for each file that was not successfully formatted. If keep_warnings is True, any successful format operations with warnings will also be returned.

class pw_presubmit.format.rust.RustfmtFormatter(tool_flags: Sequence[str] = (), **kwargs)

A formatter that runs rustfmt on files.

__init__(tool_flags: Sequence[str] = (), **kwargs)

format_file(file_path: Path) → FormatFixStatus

Formats the provided file in-place using rustfmt.

Returns:: A FormatFixStatus that contains relevant errors/warnings.

format_file_in_memory( file_path: Path, file_contents: bytes, ) → FormattedFileContents

Uses rustfmt to check the formatting of the requested file.

The file at file_path is NOT modified by this check.

Returns:: A populated pw_presubmit.format.core.FormattedFileContents that contains either the result of formatting the file, or an error message.

format_files( paths: Iterable[Path], keep_warnings: bool = True, ) → Iterator[Tuple[Path, FormatFixStatus]]

Uses rustfmt to format the specified files in-place.

Returns:: An iterator of Path and pw_presubmit.format.core.FormatFixStatus pairs for each file that was not successfully formatted. If keep_warnings is True, any successful format operations with warnings will also be returned.