Working with the pigweed.dev build system#

pigweed.dev is built with Bazel. When you want to add or remove files used by pigweed.dev, you’ll need to interact with this Bazel-based documentation generation (docgen) system.

Check out Appendix: Architecture overview for a top-down explanation of the main components of the docgen system.

Quickstart#

  1. Build the docs:

    $ bazelisk build //docs:docs

  2. Locally preview the docs:

    $ bazelisk run //docs:docs.serve

Setup#

Before you can do anything with the Bazel-based docgen system, you must complete this setup:

  1. Complete Configure your workstation for Pigweed development.

  2. Install Bazel.

Add files to the docs build#

Add files to the C/C++ API reference auto-generation system (Doxygen)#

  1. Package your headers into a filegroup:

    filegroup(
    name = "doxygen",
    srcs = [
        "public/pw_string/format.h",
        "public/pw_string/string.h",
        "public/pw_string/string_builder.h",
        "public/pw_string/utf_codecs.h",
        "public/pw_string/util.h",
    ],
)

  2. Update doxygen_srcs in //docs/BUILD.bazel to take a dependency on your new filegroup:

    filegroup(
    name = "doxygen_srcs",
    srcs = [
        # …
        "//pw_string:doxygen",
        # …
    ]
)

  1. Use a Breathe directive such as .. doxygenclass:: to pull the API reference content into a reStructuredText file.

Add files to the Python API reference auto-generation system (autodoc)#

If you see an error like this:

sphinx.errors.SphinxWarning: autodoc: failed to import module 'benchmark'
from module 'pw_rpc'; the following exception was raised:
No module named 'pw_rpc.benchmark'

It means that autodoc (the program we use to auto-generate Python API references) could not find the source code for the module that it was supposed to document. To fix this:

  1. Add your Python target as a dependency to the sphinx_build_binary rule in //docs/BUILD.bazel:

    sphinx_build_binary(
    name = "sphinx_build",
    target_compatible_with = incompatible_with_mcu(),
    deps = [
        # …
        "//pw_rpc/py:pw_rpc_benchmark",
        # …
    ],
)

Add reStructuredText files to Sphinx#

  1. Package your inputs into a sphinx_docs_library:

    load("@rules_python//sphinxdocs:sphinx_docs_library.bzl", "sphinx_docs_library")
load("//pw_build:compatibility.bzl", "incompatible_with_mcu")

sphinx_docs_library(
    name = "docs",
    srcs = [
        "docs.rst",
    ],
    prefix = "pw_elf/",
    target_compatible_with = incompatible_with_mcu(),
    visibility = ["//visibility:public"],
)

  2. Update docs in //docs/BUILD.bazel to take a dependency on your new sphinx_docs_library:

    sphinx_docs(
    name = "docs",
    # …
    deps = [
        # …
        "//pw_elf:docs",
        # …
    ]
)

  1. Add your new reStructuredText files to an existing toctree, or create a new one.

Add source code to the docs build#

Whenever possible, don’t manually write code examples in your reStructuredText (reST) docs. These code examples will bitrot over time. Instead, put your code examples in real source code that can actually be built and tested, and then use Sphinx’s literalinclude feature to insert the code example into your doc.

  1. Put your code example into a unit test:

    // examples.cc

TEST(StringExamples, BufferExample) {
  // START: BufferExample
  // …
  // END: BufferExample
}

  2. Include the code example in your reST:

    .. literalinclude:: ./examples.cc
   :language: cpp
   :dedent:
   :start-after: // START: BufferExample
   :end-before: // END: BufferExample

  3. Add the source code file to the srcs list in your sphinx_docs_library target:

    sphinx_docs_library(
    name = "docs",
    srcs = [
        # …
        "examples.cc",
        # …
    ],
)

Add images#

Images should not be checked into the Pigweed repo. See Image hosting.

Remove or change files in the docs build#

Here’s the general workflow:

  1. Remove or change files that are used in the docs build.

  2. Build the docs.

  3. When the docs build fails, Bazel’s logs will tell you what you need to do next. If Bazel’s logs aren’t informative, try some of the tips described in Debug the docs build.

You may need to do some or all of these steps:

  • In your module’s BUILD.bazel files, update these rules:

    • sphinx_docs_library targets (usually named docs)

    • filegroup targets named doxygen

  • Update //docs/BUILD.bazel.

  • redirects.

Build the docs#

$ bazelisk build //docs:docs

Watch the docs (automatically rebuild when files change)#

$ bazelisk run //:watch build //docs:docs

Tip

Try locally previewing the docs in one console tab and watching the docs in another tab.

Locally preview the docs#

$ bazelisk run //docs:docs.serve

A message like this should get printed to stdout:

Serving...
  Address: http://0.0.0.0:8000
  Serving directory: /home/kayce/pigweed/pigweed/bazel-out/k8-fastbuild/bin/docs/docs/_build/html
      url: file:///home/kayce/pigweed/pigweed/bazel-out/k8-fastbuild/bin/docs/docs/_build/html
  Server CWD: /home/kayce/.cache/bazel/_bazel_kayce/9659373b1552c281136de1c8eeb3080d/execroot/_main/bazel-out/k8-fastbuild/bin/docs/docs.serve.runfiles/_main

You can access the rendered docs at the URL that’s printed next to Address (http://0.0.0.0:8000 in the example).

List all docs sources#

Bazel builds the docs in a hermetic environment. All inputs to the docgen system must be copied into this hermetic environment. To check that you’re copying your files to the correct directory, run this command:

$ bazelisk build //docs:_docs/_sources

Debug the docs build#

When things go wrong, run this command to build the docs in a non-hermetic environment:

$ bazelisk run //docs:docs.run

Also consider tweaking these extra_opts from the sphinx_docs rule in //docs/BUILD.bazel:

  • Comment out the --silent warning to get more verbose logging output.

  • Check sphinx-build to see what other options you might want to add or remove. sphinx-build is the underlying command that the sphinx_docs Bazel rule runs.

Troubleshooting#

autodoc: failed to import module#

See Add files to the Python API reference auto-generation system (autodoc).

Appendix: Architecture overview#

The outputs of some components of the docgen system are used as inputs to other components.

flowchart LR Doxygen --> Breathe Breathe --> reST reST --> Sphinx Rust --> Sphinx Python --> Sphinx

  • Doxygen: We feed a bunch of C/C++ headers to Doxygen. Doxygen parses each header and generates XML metadata for all of the classes, functions, structs, etc. that it finds. We also publish the Doxygen-generated HTML as a separate subsite. This subsite is available at pigweed.dev/doxygen.

  • Breathe: We provide the Doxygen XML metadata to Breathe so that C/C++ API reference content can be inserted into our reStructuredText files.

  • reST: We gather up all the reStructuredText (reST) source files that are scattered across the Pigweed repository. Pigweed docs are authored in reST. We don’t use Markdown.

  • Rust: rustdoc generates Rust API reference content, similar to how Doxygen generates C/C++ API reference content. The Rust API references are output as HTML. It’s essentially a separate documentation subsite that is not integrated with the rest of pigweed.dev (yet). This subsite is available at URLs like pigweed.dev/rustdoc/pw_bytes/.

  • Python: We use Sphinx’s autodoc feature to auto-generate Python API reference content. In order for this to work, the Python modules must be listed as dependencies of the //docs:docs target.

  • Sphinx: Once all the other inputs are ready, we can use Sphinx (essentially a static site generator) to build the pigweed.dev website.

On this page