Automated analysis

The correctness and style of Pigweed’s source code is continuously verified using a suite of automated tools. We also make it easy to use the same tools to verify the code of projects using Pigweed.

Summary

On presubmit or in CI we verify Pigweed using:

  • pylint

  • mypy

  • clang-tidy

  • AddressSanitizer (asan)

  • ThreadSanitizer (tsan)

  • UndefinedBehaviorSanitizer (ubsan)

  • OSS-Fuzz

The rest of this document discusses these tools and their configuration in greater detail, and how to use them in your own project.

Analysis tools

Static analysis

PyLint

PyLint is a customizable Python linter. Pigweed complies with almost all the default checks; see .pylintrc for details. PyLint detects problems such as overly broad catch statements, unused arguments/variables, and mutable default parameter values.

For upstream Pigweed, PyLint can be run with ninja python.lint.pylint or ninja python.lint. It’s also included in a variety of presubmit steps, like static_analysis and python_checks.gn_python_check. See the Enabling analysis for your project section to learn how to run PyLint on your Pigweed-based project.

Mypy

Python 3 allows for type annotations for variables, function arguments, and return values. Most, but not all, of Pigweed’s Python code has type annotations, and these annotations have caught real bugs in code that didn’t yet have unit tests. Mypy is an analysis tool that enforces these annotations.

Mypy helps find bugs like when a string is passed into a function that expects a list of strings—since both are iterables this bug might otherwise be hard to track down.

Mypy can be run with ninja python.lint.mypy or ninja python.lint. It’s also included in a variety of presubmit steps, like static_analysis and python_checks.gn_python_check.

clang-tidy

clang-tidy is a C++ “linter” and static analysis tool. It identifies bug-prone patterns (e.g., use after move), non-idiomatic usage (e.g., creating std::unique_ptr with new rather than std::make_unique), and performance issues (e.g., unnecessary copies of loop variables).

While powerful, clang-tidy defines a very large number of checks, many of which are special-purpose (e.g., only applicable to FPGA HLS code, or code using the Abseil library) or have high false positive rates. Pigweed enables over 50 checks which are relevant to an embedded C/C++ library and have good signal-to-noise ratios. The full list of Pigweed’s checks is in .clang-tidy.

We do not currently enable the Clang Static Analyzers because they suffer from false positives, and their findings are time-consuming to manually verify.

clang-tidy can be run with ninja static_analysis or pw presubmit --step static_analysis.

Clang sanitizers

Note

Running sanitizers in presubmit for all Pigweed code is work in progress. See https://bugs.pigweed.dev/514 for details.

We run all of Pigweed’s unit tests with the additional instrumentation described in this section. For more detail about these sanitizers, see the Github documentation.

The exact configurations we use for these sanitizers are in pw_toolchain/host_clang/BUILD.gn. You can see the current status of the sanitizer builds in the Pigweed CI console, as pigweed-linux-san-*.

Fuzzers

Fuzz testing detects errors in software by providing it with randomly generated inputs. We use OSS-fuzz to continuously uncover potential vulnerabilities in Pigweed. Dashboard with Pigweed’s latest results. See the pw_fuzzer module documentation for more details.

Enabling analysis for your project

PyLint and Mypy

PyLint and Mypy can be configured to run every time your project is built by adding python.lint to your default build group. (You can also add one or both individually using python.lint.mypy and python.lint.pylint.) Likewise, these can be added to individual presubmit steps (examples). You can also directly include the python_checks.gn_python_lint presubmit step.

clang-tidy

pw_toolchain/static_analysis_toolchain.gni provides the pw_static_analysis_toolchain template that can be used to create a build group performing static analysis. See pw_toolchain documentation for more details. This group can then be added as a presubmit step using pw_presubmit.

You can place a .clang-tidy file at the root of your repository to control which checks are executed. See the clang documentation for a discussion of how the tool chooses which .clang-tidy files to apply when run on a particular source file.

Clang sanitizers

Note

This section is under construction.

Fuzzers

See the pw_fuzzer module documentation.