pw_ide CLI#
pw_ide: Code editor and IDE support for Pigweed projects
The pw_ide
command-line interface (CLI) provides an editor-independent
mechanism for managing code intelligence for Pigweed projects.
Note
Currently, the CLI only supports bootstrap projects, For Bazel projects, we recommend using the Visual Studio Code integration.
Setting up C++ code intelligence#
clangd is a language server that provides C/C++
code intelligence features to any editor that supports the language server
protocol (LSP). It uses a compilation database,
a JSON file containing the compile commands for the project. Projects that have
multiple targets and/or use multiple toolchains need separate compilation
databases for each target toolchain. pw_ide
provides tools for managing
those databases.
Assuming you have one or more compilation databases that have been generated by your build system, start with:
pw ide sync
This command will:
Find every compilation database in your build directory
Analyze each database
If a database is internally consistent (i.e., it only contains valid compile commands for a single target), it will use that database as-is for the target toolchain that database pertains to. This is the typical case for CMake builds.
Otherwise, if a database contains commands for multiple target toolchains and/or contains invalid compile commands, the database will be processed, yielding one new compilation database for each target toolchain. Those databases will be used instead of the original.
Link each target to its respective compilation database
Now, you can list the available target toolchains with:
pw ide cpp --list
Then set the target toolchain that clangd
should use with:
pw ide cpp --set <selected target name>
clangd
will now work as designed since it is configured to use a compilation
database that is consistent to just a single target toolchain.
clangd
must be run with arguments that provide the Pigweed environment paths
to the correct toolchains and sysroots. One way to do this is to launch your
editor from the terminal in an activated environment (e.g. running vim
from
the terminal), in which case nothing special needs to be done as long as your
toolchains are in the Pigweed environment or $PATH
. But if you launch your
editor outside of the activated environment (e.g. launching Visual Studio Code
from your GUI shell’s launcher), you can generate the command that invokes
clangd
with the right arguments with:
pw ide cpp --clangd-command
Setting up Python code intelligence#
Any Python language server should work well with Pigweed projects as long as it’s configured to use the Pigweed virtual environment. You can output the path to the virtual environment on your system with:
pw ide python --venv
Setting up docs code intelligence#
The esbonio language server will provide
code intelligence for reStructuredText and Sphinx. It works well with Pigweed
projects as long as it is pointed to Pigweed’s Python virtual environment. For
Visual Studio Code, simply install the esbonio extension, which will be
recommended to you after setting up pw_ide
. Once it’s installed, a prompt
will ask if you want to automatically install esbonio in your Pigweed Python
environment. After that, give esbonio some time to index, then you’re done!
Command-line interface reference#
CLI tools for pw_ide.
usage: pw ide [-h] [-o {stdout,log}] {sync,setup,cpp,python,vscode} ...
Named Arguments#
- -o, --output
Possible choices: stdout, log
where program output should go
Default: “pretty”
Sub-commands#
sync#
Setup or sync your Pigweed project IDE features.
pw ide sync [-h]
This will automatically set up your development environment with all the features that Pigweed IDE supports, with sensible defaults.
At minimum, this command will create the .pw_ide working directory and create settings files for all supported editors. Projects can define additional setup steps in .pw_ide.yaml.
When new IDE features are introduced in the future (either by Pigweed or your downstream project), you can re-run this command to set up the new features. It will not overwrite or break any of your existing configuration.
All commands will be run from the PW_PROJECT_ROOT directory
setup#
Deprecated! Please use pw ide sync.
pw ide setup [-h]
cpp#
Configure C/C++ code intelligence support.
pw ide cpp [-h] [-l] [-g] [-s TARGET] [--set-default] [-p]
[--process-files PROCESS_FILES [PROCESS_FILES ...]]
[--clangd-command] [--clangd-command-for SYSTEM]
Named Arguments#
- -l, --list
list the target toolchains available for C/C++ language analysis
Default: False
- -g, --get
print the current target toolchain used for C/C++ language analysis
Default: False
- -s, --set
set the target toolchain to use for C/C++ language server analysis
- --set-default
set the C/C++ analysis target toolchain to the default defined in pw_ide settings
Default: False
- -p, --process
process compilation databases found in directories defined in the settings file
Default: False
- --process-files, -P
process the compilation database(s) at the provided path(s)
- --clangd-command
print the command for your system that runs clangd in the activated Pigweed environment
Default: False
- --clangd-command-for
print the command for the specified system that runs clangd in the activated Pigweed environment
Code intelligence can be provided by clangd or other language servers that use the clangd compilation database format, defined at: https://clang.llvm.org/docs/JSONCompilationDatabase.html
Pigweed projects define their build configuration(s) via a build system, usually GN, Bazel, or CMake. Based on build configurations, the build system generates commands to compile each translation unit in the project. clangd uses those commands to parse the build graph and provide rich code intelligence.
Pigweed projects often target multiple devices & architectures, and use multiple compiler toolchains. As a result, there may be more than one way to compile each translation unit. Your build system ensures that it only invokes a single compiler command for each translation unit which is consistent with the toolchain and target appropriate to that build, which we refer to as a “target toolchain”.
We need to do the same thing with the compilation database that clangd uses. We handle this by:
Processing the compilation database produced the build system into multiple internally-consistent compilation databases, one for each target toolchain.
Providing commands to select which target toolchain you want to use for code analysis.
Refer to the Pigweed documentation or your build system’s documentation to learn how to produce a clangd compilation database. Once you have one, run this command to process it:
pw ide cpp --process-files {path to compile_commands.json}
When using GN and CMake, pw_ide
can search your build directory (or
other directories specified in settings) for compilation databases with:
pw ide cpp --process
You can now examine the target toolchains that are available to you:
pw ide cpp --list
… and select the target toolchain you want to use:
pw ide cpp --set host_clang
As long as your editor or language server plugin is properly configured, you will now get code intelligence features relevant to that particular target toolchain.
You can see what target toolchain is selected by running:
pw ide cpp
Whenever you switch to a target toolchain you haven’t used before, clangd will index the build, which may take several minutes. This process is not blocking, so you can take advantage of code analysis immediately even while the indexing is in progress. These indexes are cached, so you can switch between targets without re-indexing each time.
If your build configuration changes significantly (e.g. you add a new file to the project), you will need to re-process the compilation database for that change to be recognized and manifested in the target toolchain. Your target toolchain selection will not change, and your index will only need to be incrementally updated.
You can generate the clangd command your editor needs to run with:
pw ide cpp --clangd-command
If your editor uses JSON for configuration, you can export the same command in that format:
pw ide cpp --clangd-command-for json
python#
Configure Python code intelligence support.
pw ide python [-h] [--venv] [--install-editable MODULE]
Named Arguments#
- --venv
print the path to the Pigweed Python virtual environment
Default: False
- --install-editable
install a Pigweed Python module in editable mode
You can generate the path to the Python virtual environment interpreter that your editor/language server should use with:
pw ide python --venv
When working on Pigweed’s Python modules, it can be convenient to install them in editable mode to instantly realize code changes. You can do this by running:
pw ide python --install-editable pw_{module name}
Just note that running bootstrap or building will override this.
vscode#
Configure support for Visual Studio Code.
pw ide vscode [-h] [--include SETTINGS_TYPE [SETTINGS_TYPE ...]]
[--exclude SETTINGS_TYPE [SETTINGS_TYPE ...]]
[--build-extension]
Named Arguments#
- --include
update only these settings types
- --exclude
do not update these settings types
- --build-extension
build the extension from source
Default: False
This will replace your current Visual Studio Code (VSC) settings for this
project (in .vscode/settings.json
, etc.) with the following sets of
settings, in order:
The Pigweed default settings
Your project’s settings, if any (in
.vscode/pw_project_settings.json
)Your personal settings, if any (in
.vscode/pw_user_settings.json
)
In other words, settings files lower on the list can override settings defined by those higher on the list. Settings defined in the sources above are not active in VSC until they are merged and output to the current settings file by running:
pw ide vscode
Refer to the Visual Studio Code documentation for more information about these settings: https://code.visualstudio.com/docs/getstarted/settings
This command also manages VSC tasks (.vscode/tasks.json
) and extensions
(.vscode/extensions.json
). You can explicitly control which of these
settings types (“settings”, “tasks”, and “extensions”) is modified by
this command by using the --include
or --exclude
options.
Your current VSC settings will never change unless you run pw ide
commands. Since the current VSC settings are an artifact built from the
three settings files described above, you should avoid manually editing
that file; it will be replaced the next time you run pw ide vscode
. A
backup of your previous settings file will be made, and you can diff it
against the new file to see what changed.
These commands will never modify your VSC user settings, which are stored outside of the project repository and apply globally to all VSC instances.
The settings files are system-specific and shouldn’t be checked into the
repository, except for the project settings (those with pw_project_
),
which can be used to define consistent settings for everyone working on the
project.
Note that support for VSC can be disabled at the project level or the user level by adding the following to .pw_ide.yaml or .pw_ide.user.yaml respectively:
editors:
vscode: false
Likewise, it can be enabled by setting that value to true. It is enabled by default.
Configuration#
pw_ide
has a built-in default configuration, so you don’t need to create
a configuration file to get started. You can create a configuration file if you
need to override those defaults.
A project configuration can be defined in .pw_ide.yaml
in the project root.
This configuration will be checked into source control and apply to all
developers of the project. Each user can also create a .pw_ide.user.yaml
file that overrides both the default and project settings, is not checked into
source control, and applies only to that checkout of the project. All of these
files have the same schema, in which these options can be configured:
- property PigweedIdeSettings.working_dir: Path#
Path to the
pw_ide
working directory.The working directory holds C++ compilation databases and caches, and other supporting files. This should not be a directory that’s regularly deleted or manipulated by other processes (e.g. the GN
out
directory) nor should it be committed to source control.Default:
.pw_ide
- property PigweedIdeSettings.compdb_gen_cmd: str | None#
The command that should be run to generate a compilation database.
Defining this allows
pw_ide
to automatically generate a compilation database if it suspects one has not been generated yet before a sync.Default:
None
- property PigweedIdeSettings.compdb_search_paths: list[tuple[pathlib.Path, str]]#
Paths to directories to search for compilation databases.
If you’re using a build system to generate compilation databases, this may simply be your build output directory. However, you can add additional directories to accommodate compilation databases from other sources.
Entries can be just directories, in which case the default target inference pattern will be used. Or entries can be tuples of a directory and a target inference pattern. See the documentation for
target_inference
for more information.Finally, the directories can be concrete paths, or they can be globs that expand to multiple paths.
Note that relative directory paths will be resolved relative to the workspace root.
Default:
['out']
- property PigweedIdeSettings.target_inference: str#
A glob-like string for extracting a target name from an output path.
Build systems and projects have varying ways of organizing their build directory structure. For a given compilation unit, we need to know how to extract the build’s target name from the build artifact path. A simple example:
clang++ hello.cc -o host/obj/hello.cc.o
The top-level directory
host
is the target name we want. The same compilation unit might be used with another build target:gcc-arm-none-eabi hello.cc -o arm_dev_board/obj/hello.cc.o
In this case, this compile command is associated with the
arm_dev_board
target.When importing and processing a compilation database, we assume by default that for each compile command, the corresponding target name is the name of the top level directory within the build directory root that contains the build artifact. This is the default behavior for most build systems. However, if your project is structured differently, you can provide a glob-like string that indicates how to extract the target name from build artifact path.
A
*
indicates any directory, and?
indicates the directory that has the name of the target. The path is resolved from the build directory root, and anything deeper than the target directory is ignored. For example, a glob indicating that the directory two levels down from the build directory root has the target name would be expressed with*/*/?
.Note that the build artifact path is relative to the compilation database search path that found the file. For example, for a compilation database search path of
{project dir}/out
, for the purposes of target inference, the build artifact path is relative to the{project dir}/out
directory. Target inference patterns can be defined for each compilation database search path. See the documentation forcompdb_search_paths
for more information.Default:
?
- property PigweedIdeSettings.targets_include: list[str]#
The list of targets that should be enabled for code analysis.
In this case, “target” is analogous to a GN target, i.e., a particular build configuration. By default, all available targets are enabled. By adding targets to this list, you can constrain the targets that are enabled for code analysis to a subset of those that are available, which may be useful if your project has many similar targets that are redundant from a code analysis perspective.
Target names need to match the name of the directory that holds the build system artifacts for the target. For example, GN outputs build artifacts for the
pw_strict_host_clang_debug
target in a directory with that name in its output directory. So that becomes the canonical name for the target.Default:
[]
- property PigweedIdeSettings.targets_exclude: list[str]#
The list of targets that should not be enabled for code analysis.
In this case, “target” is analogous to a GN target, i.e., a particular build configuration. By default, all available targets are enabled. By adding targets to this list, you can disable/hide targets that should not be available for code analysis.
Target names need to match the name of the directory that holds the build system artifacts for the target. For example, GN outputs build artifacts for the
pw_strict_host_clang_debug
target in a directory with that name in its output directory. So that becomes the canonical name for the target.Default:
[]
- property PigweedIdeSettings.default_target: str | None#
The default target to use when calling
--set-default
.This target will be selected when
pw ide cpp --set-default
is called. You can define an explicit default target here. If that command is invoked without a default target definition,pw_ide
will try to infer the best choice of default target. Currently, it selects the target with the broadest compilation unit coverage.Default:
None
- property PigweedIdeSettings.cascade_targets: bool#
Mix compile commands for multiple targets to maximize code coverage.
By default (with this set to
False
), the compilation database for each target is consistent in the sense that it only contains compile commands for one build target, so the code intelligence that database provides is related to a single, known compilation artifact. However, this means that code intelligence may not be provided for every source file in a project, because some source files may be relevant to targets other than the one you have currently set. Those source files won’t have compile commands for the current target, and no code intelligence will appear in your editor.If this is set to
True
, compilation databases will still be separated by target, but compile commands for all other targets will be appended to the list of compile commands for that target. This will maximize code coverage, ensuring that you have code intelligence for every file that is built for any target, at the cost of consistency—the code intelligence for some files may show information that is incorrect or irrelevant to the currently selected build target.The currently set target’s compile commands will take priority at the top of the combined file, then all other targets’ commands will come after in order of the number of commands they have (i.e. in the order of their code coverage). This relies on the fact that
clangd
parses the compilation database from the top down, using the first compile command it encounters for each compilation unit.Default:
False
- property PigweedIdeSettings.sync: list[str]#
A sequence of commands to automate IDE features setup.
pw ide sync
should do everything necessary to get the project from a fresh checkout to a working default IDE experience. This defines the list of commands that makes that happen, which will be executed sequentially in subprocesses. These commands should be idempotent, so that the user can run them at any time to update their IDE features configuration without the risk of putting those features in a bad or unexpected state.Default:
['pw --no-banner ide cpp --process']
- property PigweedIdeSettings.clangd_additional_query_drivers: list[str] | None#
Additional query driver paths that clangd should use.
By default,
pw_ide
supplies driver paths for the toolchains included in Pigweed. If you are using toolchains that are not supplied by Pigweed, you should include path globs to your toolchains here. These paths will be given higher priority than the Pigweed toolchain paths.If you want to omit the query drivers argument altogether, set this to
null
.Default:
[]
- property PigweedIdeSettings.workspace_root: Path#
The root directory of the IDE workspace.
In most cases, the IDE workspace directory is also your Pigweed project root; that’s the default, and if that’s the case, this you can omit this configuration.
If your project has a structure where the directory you open in your IDE is not the Pigweed project root directory, you can specify the workspace root directory here to ensure that IDE support files are put in the right place.
For example, given this directory structure:
my_project | ├- third_party ├- docs ├- src | ├- pigweed.json | └- ... all other project source files | └- pigweed └ ... upstream Pigweed source, e.g. a submodule
In this case
`my_project/src/`
is the Pigweed project root, but`my_project/`
is the workspace root directory you will open in your IDE.A relative path will be resolved relative to the directory in which the
.pw_ide.yaml
config file is located.
When to provide additional configuration to support your use cases#
The default configuration for clangd
in pw_ide
should work without
additional configuration as long as you’re using only toolchains provided by
Pigweed and your native host toolchain. If you’re using other toolchains, keep
reading.
clangd
needs two pieces of information to use a toolchain:
A path to the compiler, which will be taken from the compile command.
The same compiler to be reflected in the query driver argument provided when running
clangd
.
When using pw_ide
with external toolchains, you only need to add a path to
the compiler to clangd_additional_query_drivers
in your project’s
pw_ide.yaml
file. When processing a compilation database, pw_ide
will
use the query driver globs to find your compiler and configure clangd
to
use it.
Using compiler wrappers#
If you’re using ccache
or any other wrapper command that is configured
using ccache
’s’ KEY=VALUE
pattern, it will work out of the box.