Static and runtime analysis#

Automated static analysis and runtime sanitizers help ensure the correctness of your project’s code. Using these tools has long been a best practice in developing software running in data centers, desktops, and mobile devices. Pigweed makes these tools easy to integrate into embedded development, too!

If you’re integrating Pigweed into an existing project using Bazel, you can get started on static and runtime analysis as soon as you’ve completed Set up Pigweed as an external dependency.

The Checklist#

These are the tools we recommend using when developing firmware with Pigweed.

Tool

Summary

Check?

clang-tidy

C++ linter and static analyzer

Pylint

Python linter

Mypy

Python type checker

Clang sanitizers

C++ runtime checks

Fuzzing

Finds bugs on random code paths

The rest of this document explains why you should use each of these tools, and how to do so.

Note

We don’t discuss CI integration in this guide, but it’s absolutely essential! If you’re using GitHub Actions, see Set up GitHub Actions for a Pigweed project for instructions.

Static analysis#

Static analysis attempts to find correctness or performance issues in software without executing it.

clang-tidy#

clang-tidy is a C++ “linter” and static analysis tool. It identifies bug-prone patterns (e.g., use after move), non-idiomatic usage (e.g., creating std::unique_ptr with new rather than std::make_unique), and performance issues (e.g., unnecessary copies of loop variables).

While powerful, clang-tidy defines a very large number of checks, many of which are special-purpose (e.g., only applicable to FPGA HLS code, or code using the Abseil library) or have high false positive rates. Here’s a list of recommended checks which are relevant to embedded C/C++ projects and have good signal-to-noise ratios. See the list of checks in clang-tidy documentation for detailed documentation for each check.

Checks: >
    bugprone-argument-comment,
    bugprone-assert-side-effect,
    bugprone-bool-pointer-implicit-conversion,
    bugprone-dangling-handle,
    bugprone-fold-init-type,
    bugprone-forwarding-reference-overload,
    bugprone-forward-declaration-namespace,
    bugprone-inaccurate-erase,
    bugprone-macro-repeated-side-effects,
    bugprone-move-forwarding-reference,
    bugprone-multiple-statement-macro,
    bugprone-string-constructor,
    bugprone-suspicious-memset-usage,
    bugprone-swapped-arguments,
    bugprone-undefined-memory-manipulation,
    bugprone-undelegated-constructor,
    bugprone-unused-raii,
    bugprone-use-after-move,
    clang-diagnostic-*,
    -clang-analyzer-*,
    darwin-avoid-spinlock,
    google-build-explicit-make-pair,
    google-build-namespaces,
    google-default-arguments,
    google-global-names-in-headers,
    google-readability-function-size,
    google-readability-namespace-comments,
    google-runtime-operator,
    misc-static-assert,
    misc-unconventional-assign-operator,
    misc-unused-using-decls,
    modernize-avoid-bind,
    modernize-deprecated-ios-base-aliases,
    modernize-make-shared,
    modernize-make-unique,
    modernize-pass-by-value,
    modernize-replace-auto-ptr,
    modernize-replace-disallow-copy-and-assign-macro,
    modernize-replace-random-shuffle,
    modernize-shrink-to-fit,
    modernize-unary-static-assert,
    modernize-use-bool-literals,
    modernize-use-emplace,
    modernize-use-equals-delete,
    modernize-use-noexcept,
    modernize-use-nullptr,
    modernize-use-override,
    modernize-use-transparent-functors,
    modernize-use-uncaught-exceptions,
    performance-faster-string-find,
    performance-for-range-copy,
    performance-implicit-conversion-in-loop,
    performance-inefficient-algorithm,
    performance-inefficient-vector-operation,
    performance-move-constructor-init,
    readability-container-size-empty,
    readability-inconsistent-declaration-parameter-name,
    readability-misleading-indentation,
    readability-redundant-control-flow,
    readability-redundant-smartptr-get,
    readability-string-compare,

We will extend this list over time, see b/234876263.

We do not currently recommend the Clang Static Analyzers because they suffer from false positives, and their findings are time-consuming to manually verify.

How to enable clang-tidy#

We recommend using bazel_clang_tidy to run clang-tidy from Bazel.

If you’re using Pigweed’s own host toolchain configuration, see the clang-tidy section for simple step-by-step instructions.

pw_toolchain/static_analysis_toolchain.gni provides the pw_static_analysis_toolchain template that can be used to create a build group performing static analysis. See pw_toolchain documentation for more details. This group can then be added as a presubmit step using pw_presubmit.

You can place a .clang-tidy file at the root of your repository to control which checks are executed. See the clang documentation for a discussion of how the tool chooses which .clang-tidy files to apply when run on a particular source file.

Pylint#

Pylint is a customizable Python linter that detects problems such as overly broad catch statements, unused arguments/variables, and mutable default parameter values.

How to enable Pylint#

Add the following to your .bazelrc file:

build:pylint --aspects @pigweed//pw_build:pw_pylint.bzl%pylint_aspect
build:pylint --output_groups=report
build:pylint --build_tag_filters=-nopylint

Run Pylint on all your Python targets:

bazelisk build --config=pylint //...

Pylint can be configured to run every time your project is built by adding python.lint.pylint to your default build group. You can also add this to a presubmit step (examples), or directly include the python_checks.gn_python_lint presubmit step.

Pylint configuration (Bazel only)#

Disabling Pylint for individual build targets#

To exempt a particular build target from being linted with Pylint, add tags = ["nopylint"] to its attributes.

Configuring the pylintrc file#

By default, Pigweed’s Pylint aspect uses an empty pylintrc file. To use a pylintrc file, set the @pigweed//pw_build:pylintrc label flag. For example, to use the .pylintrc at the root of your repository, add the following line to your .bazelrc:

build:pylint --@pigweed//pw_build:pylintrc=//:.pylintrc

You may wish to use Pigweed’s own .pylintrc as a starting point.

Configuring the Pylint binary#

By default, Pigweed’s Pylint aspect fetches Pylint from PyPI using pip, based on upstream Pigweed’s pip requirements. To use a different Pylint binary, set the @pigweed//pw_build:pylint label flag to point to the binary you wish to use.

In particular, if you also wish to use Pylint from PyPI but at a different version, create a py_console_script_binary to wrap it and point the label flag to it.

Known limitations#

  • The wrong-import-order check is unsupported. Because the directory layout of Python files in the Bazel sandbox is different from a regular Python venv, Pylint is confused about which imports are first- versus third-party. This seems hard to fix because Pylint uses undocumented heuristics to categorize the imports.

Mypy#

Python 3 allows for type annotations for variables, function arguments, and return values. Most of Pigweed’s Python code has type annotations, and these annotations have caught real bugs in code that didn’t yet have unit tests. Mypy is an analysis tool that enforces these annotations.

Mypy helps find bugs like when a string is passed into a function that expects a list of strings—since both are iterables this bug might otherwise be hard to track down.

How to enable Mypy#

We recommend using rules_mypy for running Mypy from Bazel.

Examples of enabling Mypy in existing projects that may be helpful:

  • Sense, for a downstream project without pip dependencies.

  • Upstream Pigweed, for a library with pip dependencies.

Similarly to Pylint, Mypy can be configured to run every time your project is built by adding python.lint.mypy to your default build group.

Clang sanitizers#

Clang sanitizers detect serious errors common in C/C++ programs.

Note

Pigweed does not yet support MemorySanitizer (msan). See b/234876100 for details.

Unlike clang-tidy, the clang sanitizers are runtime instrumentation: the instrumented binary needs to be run for issues to be detected. We recommend you run all of your project’s host unit tests (and the host ports of your application code) with all these sanitizers.

Unfortunately, the different sanitizers cannot generally be enabled at the same time. We recommend that in CI you run all of your tests three times: once with asan, again with ubsan, and again with tsan.

How to enable the sanitizers#

Sanitizers in Bazel#

If you’re using Pigweed’s own host toolchain configuration, you can enable asan, tsan, and ubsan by building with the appropriate flags:

bazelisk build --@pigweed//pw_toolchain/host_clang:asan //...
bazelisk build --@pigweed//pw_toolchain/host_clang:ubsan //...
bazelisk build --@pigweed//pw_toolchain/host_clang:tsan //...

If you’re building your own clang-based toolchain, you can add to it @pigweed//pw_toolchain/cc/args:asan (and analogous flags for tsan and ubsan).

Sanitizers in GN#

There are two ways to enable sanitizers for your build.

GN args on debug toolchains#

If you are already building your tests with one of the following toolchains (or a toolchain derived from one of them):

  • pw_toolchain_host_clang.debug

  • pw_toolchain_host_clang.speed_optimized

  • pw_toolchain_host_clang.size_optimized

You can enable the clang sanitizers simply by setting the GN arg pw_toolchain_SANITIZERS to the desired subset of ["address", "thread", "undefined"]. For example, if you only want to run asan and ubsan, then your arg value should be ["address", "undefined"].

Example#

If your project defines a toolchain host_clang_debug that is derived from one of the above toolchains, and you’d like to run the pw_executable target sample_binary defined in the BUILD.gn file in examples/sample with asan, you would run this:

gn gen out --args='pw_toolchain_SANITIZERS=["address"]'
ninja -C out host_clang_debug/obj/example/sample/bin/sample_binary
out/host_clang_debug/obj/example/sample/bin/sample_binary
Sanitizer toolchains#

Otherwise, instead of using gn args you can build your tests with the appropriate toolchain from the following list (or a toolchain derived from one of them):

  • pw_toolchain_host_clang.asan

  • pw_toolchain_host_clang.ubsan

  • pw_toolchain_host_clang.tsan

See the pw_toolchain module documentation for more about Pigweed toolchains.

Fuzzers#

See the pw_fuzzer module documentation.

On this page