Website updates#

This page discusses how to make frontend and backend website changes to pigweed.dev, Pigweed’s main documentation website.

Overview#

The key thing to understand is that pigweed.dev is actually 3 completely separate documentation websites combined together. The C/C++ API reference is generated with Doxygen. The Rust API reference is generated with rustdoc. Everything else is generated with Sphinx. We use Bazel to ensure that the rustdoc and Doxygen builds finish before the Sphinx build and are available as inputs to Sphinx. We then use Sphinx extensions to glue the 3 sites together into one cohesive whole.

Life of a docs build#

A brief explanation of what happens when you run bazelisk build //docs:

  1. Bazel resolves the //docs alias to //docs/sphinx:docs and attempts to build that target. I.e. it attempts to run the Sphinx build.

  2. Bazel detects that it must actually run the Doxygen and rustdoc builds first because the //docs/sphinx:docs target lists //docs/doxygen:html and //pw_rust:docs as deps. When these targets finish building, the complete Doxygen and rustdoc subsites are essentially provided as inputs to the Sphinx build.

  3. The Sphinx build runs. Sphinx invokes our extensions. These extensions inspect and mutate Doxygen, rustdoc, and Sphinx information in order to glue the 3 subsites together.

Docs deployments#

How do docs updates get deployed to pigweed.dev? When a new commit merges, we have a CI pipeline that runs bazelisk build //docs and uploads the built site to a Google Cloud Storage bucket. A Google App Engine server runs the production site. All of the docs deployment workflow is closed source.

Image hosting#

Images should not be checked into the Pigweed repository because it significantly slows down the repository cloning process. Images should instead be hosted on Pigweed’s image CDN, https://www.gstatic.com/pigweed/.

If you’re adding an image to a pigweed.dev doc, here’s the recommended workflow:

  1. When drafting a change, it’s OK to temporarily check in the image so that there is a record of it in Gerrit.

  2. When your change is almost ready to merge, a Pigweed teammate will upload your image to Pigweed’s image CDN, and then leave a comment on your change asking you to delete the checked-in image and replace the reference to it with the URL to the CDN-hosted image.

Google employees working with external contributors should go to go/pigweed-gstatic to upload images.

Create redirects#

pigweed.dev supports client-side HTML redirects. The redirects are powered by sphinx-reredirects.

To create a redirect:

  1. Open //docs/sphinx/redirects.json.

  1. Create a new key-value pair. The key is the obsolete path that should be redirected. The value is the redirect destination. See Usage.

    • The path in the key should not have a filename extension. The path in the value should.

    • The path in the value should be relative to the path in the key.

    • The path in the value should contain the full HTML filename. E.g. values should be like ./examples/index.html, not ./examples/.

Example of the redirect that sphinx-reredirects auto-generates:

<html>
  <head>
    <meta http-equiv="refresh" content="0; url=pw_sys_io_rp2040/docs.html">
  </head>
</html>

Note

Server-side redirects are the most robust solution, but client-side redirects are good enough for our needs:

  • Client-side redirects are supported in all browsers and should therefore work for all real pigweed.dev readers.

  • Client-side redirects were much easier and faster to implement.

  • Client-side redirects can be stored alongside the rest of the pigweed.dev source code.

  • Google Search interprets the kind of client side redirects that we use as permanent redirects, which is the behavior we want. See meta refresh and its HTTP equivalent. The type of client-side redirect we used is called a “instant meta refresh redirect” in that guide.

Copy-to-clipboard feature on code blocks#

The copy-to-clipboard feature on code blocks is powered by sphinx-copybutton.

sphinx-copybutton recognizes $ as an input prompt and automatically removes it.

There is a workflow for manually removing the copy-to-clipboard button for a particular code block but it has not been implemented yet. See Remove copybuttons using a CSS selector.

Fonts and typography#

pigweed.dev is taking an iterative approach to its fonts and typography. See b/353530954 for context, examples of how to update fonts, and to leave feedback.

Rationale for current choices:

  • Headings: Lato. Per UX team’s recommendation.

  • Copy: Noto Sans. Noto is one of two fonts recommended by Material Design 3. It seems to complement Lato well. See Typography.

  • Code: Roboto Mono. Also per UX team’s recommendation. Roboto Mono is mature and well-established in this space.

Sitemap generation#

https://pigweed.dev/sitemap.xml is generated by the custom Sphinx Extension located at //docs/sphinx/_extensions/sitemap.py. A custom extension is necessary because the pigweed.dev production server redirects pages that end in …/docs.html to …/ (e.g. pigweed.dev/pw_string/docs.html redirects to pigweed.dev/pw_string/) and no third-party extension supports the kind of URL rewrite customization that we need. See b/386257958.

Google Analytics ID#

The pigweed.dev Google Analytics ID is not hardcoded anywhere in the upstream Pigweed repo. It is passed through the environment like this:

  1. Docs builders provide a Google Analytics ID as a command line argument.

  2. //docs/sphinx/conf.py looks for the existence of a GOOGLE_ANALYTICS_ID OS environment variable and passes the variable along to Sphinx when found.

  3. //pw_docgen/py/pw_docgen/sphinx/google_analytics.py looks for the Sphinx build environment variable and injects the ID (and related JavaScript code) into each page’s HTML when found.

Passing the ID through the environment helps us ensure that the production ID is only used when someone views the docs from the production domain (pigweed.dev).

Testing#

All documentation tests are located in //docs/tests. The directory is a standalone Bazel workspace because the integration tests are slow and pull in heavy dependencies e.g. Playwright.

There are 2 types of tests:

  • Static tests inspect the built HTML files directly without running a browser. E.g. verifying that every page contains a <pw-header> element.

  • Runtime tests launch a headless Chromium browser instance via Playwright to test interactive UI behavior. E.g. verifying that searching via Pagefind navigates to expected results.

Quickstart#

  1. Build the docs:

    bazelisk build //docs

  2. cd into the tests directory:

    cd docs/tests

  3. Run all tests:

    bazelisk test --test_output=all //...

    Or run an individual test e.g. header_test:

    bazelisk test --test_output=all //:header_test

Universal header#

As mentioned in Overview, pigweed.dev is actually 3 separate sites glued together: Sphinx, rustdoc, and Doxygen. Yet as you browse the website there is a consistent header at the top of all pages. How does that work?

It’s essentially postprocessing. We inject an HTML comment (<!-- pw-sentinel -->) into every page. At the end of the Sphinx build, one of our Sphinx extensions replaces this sentinel comment with the header HTML, CSS, and JS. We also have to override a lot of the default Doxygen and rustdoc CSS styling in order to make our custom header look correct on those subsites. All of the header customization code is located at //docs/common/header.*.

Client-side URL rewriting#

pigweed.dev documentation is published in various deployment environments:

  • Production site: Rooted at https://pigweed.dev/

  • Staging builds: Rooted at arbitrary subpaths (e.g. https://storage.googleapis.com/pigweed-docs-try/8673174856411998625/index.html)

  • Local preview servers: Rooted at custom ports or local filesystem paths (e.g. http://localhost:8000/ or file:///path/to/docs/out/)

Because navigation links cannot assume that the root of the website is always / or /index.html, absolute links like https://pigweed.dev/pw_string/ would break staging and local preview workflows by navigating users away to the production site.

To solve this, //docs/common/dev.js dynamically inspects the current document’s relative root path (derived from Sphinx’s DOCUMENT_NAME or data-content_root attribute) and rewrites absolute production URLs on page load so that links resolve relative to the active server’s root.

Theme selector#

The universal site header includes a light/dark theme switcher component (<pw-theme>).

The theme selector component is defined in //docs/common/header.js and //docs/common/header.css, with root-level color scheme variables defined in //docs/common/theme.css. It automatically syncs with the user’s operating system color scheme preference and persists user selections in localStorage.