pw_metric#

Stable

Overview#

Pigweed’s metric module is a lightweight manual instrumentation system for tracking system health metrics like counts or set values. For example, pw_metric could help with tracking the number of I2C bus writes, or the number of times a buffer was filled before it could drain in time, or safely incrementing counters from ISRs.

Key features of pw_metric:

Tokenized names - Names are tokenized using the pw_tokenizer enabling long metric names that don’t bloat your binary.
Tree structure - Metrics can form a tree, enabling grouping of related metrics for clearer organization.
Per object collection - Metrics and groups can live on object instances and be flexibly combined with metrics from other instances.
Global registration - For legacy code bases or just because it’s easier, pw_metric supports automatic aggregation of metrics. This is optional but convenient in many cases.
Simple design - There are only two core data structures: Metric and Group, which are both simple to understand and use. The supported metric types are uint32_t, float, and uint64_t. This module does not support complicated aggregations like running average or min/max.

Example: Instrumenting a single object#

The below example illustrates what instrumenting a class with a metric group and metrics might look like. In this case, the object’s MySubsystem::metrics() member is not globally registered; the user is on their own for combining this subsystem’s metrics with others.

#include "pw_metric/metric.h"

class MySubsystem {
 public:
  void DoSomething() {
    attempts_.Increment();
    if (ActionSucceeds()) {
      successes_.Increment();
    }
  }
  Group& metrics() { return metrics_; }

 private:
  PW_METRIC_GROUP(metrics_, "my_subsystem");
  PW_METRIC(metrics_, attempts_, "attempts", 0u);
  PW_METRIC(metrics_, successes_, "successes", 0u);
};

The metrics subsystem has no canonical output format at this time, but a JSON dump might look something like this:

{
  "my_subsystem" : {
    "successes" : 1000,
    "attempts" : 1200,
  }
}

In this case, every instance of MySubsystem will have unique counters.

Example: Instrumenting a legacy codebase#

A common situation in embedded development is debugging legacy code or code which is hard to change; where it is perhaps impossible to plumb metrics objects around with dependency injection. The alternative to plumbing metrics is to register the metrics through a global mechanism. pw_metric supports this use case. For example:

Before instrumenting:

// This code was passed down from generations of developers before; no one
// knows what it does or how it works. But it needs to be fixed!
void OldCodeThatDoesntWorkButWeDontKnowWhy() {
  if (some_variable) {
    DoSomething();
  } else {
    DoSomethingElse();
  }
}

After instrumenting:

#include "pw_metric/global.h"
#include "pw_metric/metric.h"

PW_METRIC_GLOBAL(legacy_do_something, "legacy_do_something");
PW_METRIC_GLOBAL(legacy_do_something_else, "legacy_do_something_else");

// This code was passed down from generations of developers before; no one
// knows what it does or how it works. But it needs to be fixed!
void OldCodeThatDoesntWorkButWeDontKnowWhy() {
  if (some_variable) {
    legacy_do_something.Increment();
    DoSomething();
  } else {
    legacy_do_something_else.Increment();
    DoSomethingElse();
  }
}

In this case, the developer merely had to add the metrics header, define some metrics, and then start incrementing them. These metrics will be available globally through the pw::metric::global_metrics object defined in pw_metric/global.h.

Why not just use simple counter variables?#

One might wonder what the point of leveraging a metric library is when it is trivial to make some global variables and print them out. There are a few reasons:

Metrics offload - To make it easy to get metrics off-device by sharing the infrastructure for offloading.
Consistent format - To get the metrics in a consistent format (e.g. protobuf or JSON) for analysis
Uncoordinated collection - To provide a simple and reliable way for developers on a team to all collect metrics for their subsystems, without having to coordinate to offload. This could extend to code in libraries written by other teams.
Pre-boot or interrupt visibility - Some of the most challenging bugs come from early system boot when not all system facilities are up (e.g. logging or UART). In those cases, metrics provide a low-overhead approach to understand what is happening. During early boot, metrics can be incremented, then after boot dumping the metrics provides insights into what happened. While basic counter variables can work in these contexts too, one still has to deal with the offloading problem; which the library handles.

Metrics API reference#

The metrics API consists of just a few components:

The core data structures pw::metric::Metric and pw::metric::Group
The macros for scoped metrics and groups PW_METRIC and PW_METRIC_GROUP
The macros for globally registered metrics and groups PW_METRIC_GLOBAL and PW_METRIC_GROUP_GLOBAL
The global groups and metrics list: pw::metric::global_groups and pw::metric::global_metrics.

Metric architecture#

To support both 32-bit and 64-bit metrics efficiently without virtual table overhead or wasting memory, pw_metric uses a type-discriminated class hierarchy:

Class	Description	Size (32-bit platform)
`UntypedMetric`	Common base class stored in intrusive lists. Holds the token and type.	8 bytes
`TypedMetric<uint32_t>`	Inherits from `UntypedMetric`, holds a 32-bit `uint32_t` payload.	12 bytes
`TypedMetric<float>`	Inherits from `UntypedMetric`, holds a 32-bit `float` payload.	12 bytes
`TypedMetric<uint64_t>`	Inherits from `UntypedMetric`, holds a 64-bit `uint64_t` payload (optional).	16 bytes
`TypedMetric<int64_t>`	Inherits from `UntypedMetric`, holds a 64-bit `int64_t` payload (optional).	16 bytes

All mutation operations on metrics are atomic, requiring a std::atomic backend.

64-bit atomics and toolchain support#

To support 64-bit metrics (uint64_t), the target architecture and toolchain must support 64-bit atomic operations.

Some 32-bit ARM Cortex-M targets (specifically Cortex-M0, M0+, M3, and M4) do not support 64-bit atomic operations in hardware (Cortex-M0 and M0+ lack atomic hardware instructions entirely, while Cortex-M3 and M4 lack double-word exclusive instructions like LDREXD/STREXD). When compiling for these targets, the GNU Arm Embedded Toolchain (arm-none-eabi-gcc) will generate calls to runtime helper functions (e.g., __atomic_load_8, __atomic_store_8, etc.), which are not provided by the compiler’s default runtime libraries (libgcc).

Opting-in to 64-bit metrics#

By default, 64-bit metrics are disabled on target platforms to prevent compilation and linker failures on architectures without native 64-bit atomic instructions. On host platforms (like x86_64 and aarch64), they are enabled by default.

To opt-in to 64-bit metrics on a target platform:

Define the configuration macro: Define PW_METRIC_CONFIG_ENABLE_64BIT=1 in your compiler options or configuration header.
Link the atomic helper library: Link the cortex_m_64bit_atomics library provided by Pigweed (see below).
Include the 64-bit header: Include pw_metric/metric_64bit.h in source files that define or use 64-bit metrics.

Linking the atomic helpers#

GN: Add $dir_pw_atomic:cortex_m_64bit_atomics to your target’s link dependencies.
Bazel: Add @pigweed//pw_atomic:cortex_m_64bit_atomics to your binary target’s deps.
CMake: Link the pw_atomic.cortex_m_64bit_atomics library to your binary target.

Token and type encoding#

To minimize memory usage, the metric name (token) and its type are packed into a single 32-bit field (name_and_type_) in UntypedMetric:

Lower 28 bits: The tokenized name (masked with 0x0fffffff).
Upper 4 bits: The type discriminator (masked with 0xf0000000).

This 4-bit discriminator allows up to 16 types, currently supporting: - kTypeUint32 (0x00000000) - kTypeFloat (0x10000000) - kTypeUint64 (0x20000000) (only when 64-bit is enabled) - kTypeInt64 (0x30000000) (only when 64-bit is enabled)

The remaining slots are reserved for future types (e.g., int32_t, double).

Using a 28-bit token still provides a very large token space (over 268 million possible values), keeping the probability of a name collision extremely low (e.g., less than 0.05% for 500 metrics), with build-time collision detection provided by pw_tokenizer.

TypedMetric#

The primary way to interact with metrics is through the pw::metric::TypedMetric wrappers, which are instantiated by the PW_METRIC macros.

template<typename T> class pw::metric::TypedMetric#: Type-safe wrapper for metrics. Only the operations valid for the type T are exposed at compile-time.

template<> class pw::metric::TypedMetric<uint32_t> : public pw::metric::UntypedMetric#

void Increment(uint32_t amount = 1)#: Atomically increment the metric by the given amount (saturating to max).

void Decrement(uint32_t amount = 1)#: Atomically decrement the metric by the given amount (saturating to 0).

void Set(uint32_t value)#: Atomically set the metric to the given value.

uint32_t value() const#: Return the current value of the metric.

template<> class pw::metric::TypedMetric<uint64_t> : public pw::metric::UntypedMetric#

void Increment(uint64_t amount = 1)#: Atomically increment the metric by the given amount (saturating to max).

void Decrement(uint64_t amount = 1)#: Atomically decrement the metric by the given amount (saturating to 0).

void Set(uint64_t value)#: Atomically set the metric to the given value.

uint64_t value() const#: Return the current value of the metric.

template<> class pw::metric::TypedMetric<int64_t> : public pw::metric::UntypedMetric#

void Increment(int64_t amount = 1)#: Atomically increment the metric by the given amount (saturating to max).

void Decrement(int64_t amount = 1)#: Atomically decrement the metric by the given amount (saturating to min).

void Set(int64_t value)#: Atomically set the metric to the given value.

int64_t value() const#: Return the current value of the metric.

template<> class pw::metric::TypedMetric<float> : public pw::metric::UntypedMetric#

void Set(float value)#: Atomically set the metric to the given value.

float value() const#: Return the current value of the metric.

Note

TypedMetric<float> does not offer Increment() or Decrement() operations, as floating-point addition is prone to precision loss when accumulating small values.

Group#

The pw::metric::Group object is simply:

A name for the group
A list of children groups
A list of leaf metrics groups
A 32-bit next pointer (intrusive list)

The group object is 16 bytes on 32-bit platforms.

class pw::metric::Group#

Dump(int indent_level = 0)#

Recursively dump a metrics group to pw_log. Produces output like:

"$6doqFw==": {
  "$05OCZw==": {
    "$VpPfzg==": 1,
    "$LGPMBQ==": 1.000000,
    "$+iJvUg==": 5,
  }
  "$9hPNxw==": 65,
  "$oK7HmA==": 13,
  "$FCM4qQ==": 0,
}

Note the metric names are tokenized with base64. Decoding requires using the Pigweed detokenizer. With a detokenizing-enabled logger, you could get something like:

"i2c_1": {
  "gyro": {
    "num_sampleses": 1,
    "init_time_us": 1.000000,
    "initialized": 5,
  }
  "bus_errors": 65,
  "transactions": 13,
  "bytes_sent": 0,
}

Macros#

The macros are the primary mechanism for creating metrics, and should be used instead of directly constructing metrics or groups. The macros handle tokenizing the metric and group names.

PW_METRIC(identifier, name, value)#

PW_METRIC(group, identifier, name, value)#

PW_METRIC_STATIC(identifier, name, value)#

PW_METRIC_STATIC(group, identifier, name, value)#

Declare a metric, optionally adding it to a group.

identifier - An identifier name for the created variable or member. For example: i2c_transactions might be used as a local or global metric; inside a class, could be named according to members (i2c_transactions_ for Google’s C++ style).
name - The string name for the metric. This will be tokenized. There are no restrictions on the contents of the name; however, consider restricting these to be valid C++ identifiers to ease integration with other systems.
value - The initial value for the metric. Must be either a floating point value (e.g. 3.2f) or unsigned int (e.g. 21u).
group - A pw::metric::Group instance. If provided, the metric is added to the given group.

The macro declares a variable or member named “name” with type pw::metric::Metric, and works in three contexts: global, local, and member.

If the _STATIC variant is used, the macro declares a variable with static storage. These can be used in function scopes, but not in classes.

At global scope:

PW_METRIC(foo, "foo", 15.5f);

void MyFunc() { foo.Increment(); }

At local function or member function scope:

void MyFunc() {
  PW_METRIC(foo, "foo", 15.5f);
  foo.Increment();
  // foo goes out of scope here; be careful!
}

At member level inside a class or struct:

struct MyStructy {
  void DoSomething() { somethings.Increment(); }
  // Every instance of MyStructy will have a separate somethings counter.
  PW_METRIC(somethings, "somethings", 0u);
}

You can also put a metric into a group with the macro. Metrics can belong to strictly one group, otherwise an assertion will fail. Example:

PW_METRIC_GROUP(my_group, "my_group");
PW_METRIC(my_group, foo, "foo", 0.2f);
PW_METRIC(my_group, bar, "bar", 44000u);
PW_METRIC(my_group, zap, "zap", 3.14f);

Tip

If you want a globally registered metric, see pw_metric/global.h; in that context, metrics are globally registered without the need to centrally register in a single place.

PW_METRIC_TYPED(identifier, name, type, value)#

PW_METRIC_TYPED(group, identifier, name, type, value)#

PW_METRIC_TYPED_STATIC(identifier, name, type, value)#

PW_METRIC_TYPED_STATIC(group, identifier, name, type, value)#

Declare a metric with an explicit type, optionally adding it to a group.

This macro works identically to PW_METRIC, but requires explicitly specifying the metric type (e.g. float, uint32_t, or uint64_t). This is useful for declaring 64-bit metrics or when you want to make the metric’s type explicit.

Example:

PW_METRIC_GROUP(my_group, "my_group");
PW_METRIC_TYPED(my_group, my_64bit_metric, "my_64bit_metric", uint64_t, 0ULL);

PW_METRIC_GROUP(identifier, name)#

PW_METRIC_GROUP(parent_group, identifier, name)#

PW_METRIC_GROUP_STATIC(identifier, name)#

PW_METRIC_GROUP_STATIC(parent_group, identifier, name)#

Declares a pw::metric::Group with name name; the name is tokenized. Works similar to PW_METRIC and can be used in the same contexts (global, local, and member). Optionally, the group can be added to a parent group.

If the _STATIC variant is used, the macro declares a variable with static storage. These can be used in function scopes, but not in classes.

Example:

PW_METRIC_GROUP(my_group, "my_group");
PW_METRIC(my_group, foo, "foo", 0.2f);
PW_METRIC(my_group, bar, "bar", 44000u);
PW_METRIC(my_group, zap, "zap", 3.14f);

PW_METRIC_GLOBAL(identifier, name, value)#

Declare a pw::metric::Metric with name name, and register it in the global metrics list pw::metric::global_metrics.

Example:

#include "pw_metric/global.h"
#include "pw_metric/metric.h"

// No need to coordinate collection of foo and bar; they're autoregistered.
PW_METRIC_GLOBAL(foo, "foo", 0.2f);
PW_METRIC_GLOBAL(bar, "bar", 44000u);

Note that metrics defined with PW_METRIC_GLOBAL should never be added to groups defined with PW_METRIC_GROUP_GLOBAL. Each metric can only belong to one group, and metrics defined with PW_METRIC_GLOBAL are pre-registered with the global metrics list.

Attention

Do not create PW_METRIC_GLOBAL instances anywhere other than global scope. Putting these on an instance (member context) would lead to dangling pointers and misery. Metrics are never deleted or unregistered!

PW_METRIC_GLOBAL_TYPED(identifier, name, type, value)#

Declare a pw::metric::TypedMetric with name name and explicit type, and register it in the global metrics list pw::metric::global_metrics.

This macro works identically to PW_METRIC_GLOBAL, but requires explicitly specifying the metric type (e.g. float, uint32_t, or uint64_t).

Example:

#include "pw_metric/global.h"
#include "pw_metric/metric.h"

PW_METRIC_GLOBAL_TYPED(my_global_64bit, "my_global_64bit", uint64_t, 0ULL);

PW_METRIC_GROUP_GLOBAL(identifier, name)#

Declare a pw::metric::Group with name name, and register it in the global metric groups list pw::metric::global_groups.

Note that metrics created with PW_METRIC_GLOBAL should never be added to groups! Instead, just create a freestanding metric and register it into the global group (like in the example below).

Example:

#include "pw_metric/global.h"
#include "pw_metric/metric.h"

// No need to coordinate collection of this group; it's globally registered.
PW_METRIC_GROUP_GLOBAL(legacy_system, "legacy_system");
PW_METRIC(legacy_system, foo, "foo", 0.2f);
PW_METRIC(legacy_system, bar, "bar", 44000u);

Attention

Do not create PW_METRIC_GROUP_GLOBAL instances anywhere other than global scope. Putting these on an instance (member context) would lead to dangling pointers and misery. Metrics are never deleted or unregistered!

PW_METRIC_TOKEN(name)#

Declare a pw::metric::Token (pw::tokenizer::Token) for a metric with name name. This token matches the .name() of a metric with the same name.

This is a wrapper around PW_TOKENIZE_STRING_MASK and carries the same semantics.

This is unlikely to be used by most pw_metric consumers.

Usage & Best Practices#

This library makes several tradeoffs to enable low memory use per-metric, and one of those tradeoffs results in requiring care in constructing the metric trees.

Use the Init() pattern for static objects with metrics#

A common pattern in embedded systems is to allocate many objects globally, and reduce reliance on dynamic allocation (or eschew malloc entirely). This leads to a pattern where rich/large objects are statically constructed at global scope, then interacted with via tasks or threads. For example, consider a hypothetical global Uart object:

class Uart {
 public:
  Uart(span<std::byte> rx_buffer, span<std::byte> tx_buffer)
      : rx_buffer_(rx_buffer), tx_buffer_(tx_buffer) {}

  // Send/receive here...

 private:
  pw::span<std::byte> rx_buffer;
  pw::span<std::byte> tx_buffer;
};

std::array<std::byte, 512> uart_rx_buffer;
std::array<std::byte, 512> uart_tx_buffer;
Uart uart1(uart_rx_buffer, uart_tx_buffer);

Through the course of building a product, the team may want to add metrics to the UART to for example gain insight into which operations are triggering lots of data transfer. When adding metrics to the above imaginary UART object, one might consider the following approach:

class Uart {
 public:
  Uart(span<std::byte> rx_buffer,
       span<std::byte> tx_buffer,
       Group& parent_metrics)
      : rx_buffer_(rx_buffer), tx_buffer_(tx_buffer) {
    // PROBLEM! parent_metrics may not be constructed if it's a reference
    // to a static global.
    parent_metrics.Add(tx_bytes_);
    parent_metrics.Add(rx_bytes_);
  }

  // Send/receive here which increment tx/rx_bytes.

 private:
  pw::span<std::byte> rx_buffer;
  pw::span<std::byte> tx_buffer;

  PW_METRIC(tx_bytes_, "tx_bytes", 0);
  PW_METRIC(rx_bytes_, "rx_bytes", 0);
};

PW_METRIC_GROUP(global_metrics, "/");
PW_METRIC_GROUP(global_metrics, uart1_metrics, "uart1");

std::array<std::byte, 512> uart_rx_buffer;
std::array<std::byte, 512> uart_tx_buffer;
Uart uart1(uart_rx_buffer, uart_tx_buffer, uart1_metrics);

However, this is incorrect, since the parent_metrics (pointing to uart1_metrics in this case) may not be constructed at the point of uart1 getting constructed. Thankfully in the case of pw_metric this will result in an assertion failure (or it will work correctly if the constructors are called in a favorable order), so the problem will not go unnoticed. Instead, consider using the Init() pattern for static objects, where references to dependencies may only be stored during construction, but no methods on the dependencies are called.

Instead, the Init() approach separates global object construction into two phases: The constructor where references are stored, and a Init() function which is called after all static constructors have run. This approach works correctly, even when the objects are allocated globally:

class Uart {
 public:
  // Note that metrics is not passed in here at all.
  Uart(span<std::byte> rx_buffer, span<std::byte> tx_buffer)
      : rx_buffer_(rx_buffer), tx_buffer_(tx_buffer) {}

  // Precondition: parent_metrics is already constructed.
  void Init(Group& parent_metrics) {
    parent_metrics.Add(tx_bytes_);
    parent_metrics.Add(rx_bytes_);
  }

  // Send/receive here which increment tx/rx_bytes.

 private:
  pw::span<std::byte> rx_buffer;
  pw::span<std::byte> tx_buffer;

  PW_METRIC(tx_bytes_, "tx_bytes", 0);
  PW_METRIC(rx_bytes_, "rx_bytes", 0);
};

PW_METRIC_GROUP(root_metrics, "/");
PW_METRIC_GROUP(root_metrics, uart1_metrics, "uart1");

std::array<std::byte, 512> uart_rx_buffer;
std::array<std::byte, 512> uart_tx_buffer;
Uart uart1(uart_rx_buffer, uart_tx_buffer);

void main() {
  // uart1_metrics is guaranteed to be initialized by this point, so it is
  safe to pass it to Init().uart1.Init(uart1_metrics);
}

Attention

Be extra careful about static global metric registration. Consider using the Init() pattern.

Metric member order matters in objects#

The order of declaring in-class groups and metrics matters if the metrics are within a group declared inside the class. For example, the following class will work fine:

#include "pw_metric/metric.h"

class PowerSubsystem {
 public:
  Group& metrics() { return metrics_; }
  const Group& metrics() const { return metrics_; }

 private:
  PW_METRIC_GROUP(metrics_, "power");  // Note metrics_ declared first.
  PW_METRIC(metrics_, foo, "foo", 0.2f);
  PW_METRIC(metrics_, bar, "bar", 44000u);
};

but the following one will not since the group is constructed after the metrics (and will result in a compile error):

#include "pw_metric/metric.h"

class PowerSubsystem {
 public:
  Group& metrics() { return metrics_; }
  const Group& metrics() const { return metrics_; }

 private:
  PW_METRIC(metrics_, foo, "foo", 0.2f);
  PW_METRIC(metrics_, bar, "bar", 44000u);
  PW_METRIC_GROUP(metrics_, "power");  // Error: metrics_ must be first.
};

Attention

Put groups before metrics when declaring metrics members inside classes.

Thread safety#

pw_metric has no built-in synchronization for manipulating the tree structure. Users are expected to either rely on shared global mutex when constructing the metric tree, or do the metric construction in a single thread (e.g. a boot/init thread). The same applies for destruction, though we do not advise destructing metrics or groups.

Individual metrics have atomic Increment(), Set(), and the value accessors as_float() and as_int() which don’t require separate synchronization, and can be used from ISRs.

Attention

You must synchronize access to metrics. pw_metrics does not internally synchronize access during construction. Metric Set/Increment are safe.

Lifecycle#

Metric objects are not designed to be destructed, and are expected to live for the lifetime of the program or application. If you need dynamic creation/destruction of metrics, pw_metric does not attempt to cover that use case. Instead, pw_metric covers the case of products with two execution phases:

A boot phase where the metric tree is created.
A run phase where metrics are collected. The tree structure is fixed.

Technically, it is possible to destruct metrics provided care is taken to remove the given metric (or group) from the list it’s contained in. However, there are no helper functions for this, so be careful.

Below is an example that is incorrect. Don’t do what follows!

#include "pw_metric/metric.h"

void main() {
  PW_METRIC_GROUP(root, "/");
  {
    // BAD! The metrics have a different lifetime than the group.
    PW_METRIC(root, temperature, "temperature_f", 72.3f);
    PW_METRIC(root, humidity, "humidity_relative_percent", 33.2f);
  }
  // OOPS! root now has a linked list that points to the destructed
  // "humidity" object.
}

Attention

Don’t destruct metrics. Metrics are designed to be registered / structured upfront, then manipulated during a device’s active phase. They do not support destruction.

Exporting metrics#

Collecting metrics on a device is not useful without a mechanism to export those metrics for analysis and debugging. pw_metric offers optional RPC service libraries (:metric_service_nanopb based on nanopb, and :metric_service_pwpb based on pw_protobuf) that enable exporting a user-supplied set of on-device metrics via RPC. This facility is intended to function from the early stages of device bringup through production in the field.

The MetricService provides two distinct methods for retrieving metrics, each suited for different transport characteristics.

MetricService.Get (Server Streaming) This method uses a server-streaming RPC to send all registered metrics to the caller in batches. This approach is straightforward and works well on reliable, synchronous transports where the client is always ready to receive data.

MetricService.Walk (Unary) The unary Walk RPC offers a client-driven, paginated mechanism. This is suitable for asynchronous transports where the server cannot guarantee the transport is ready, and for large metric collections that may not fit within a transport’s MTU. This is the recommended method for metric retrieval.

The metrics are fetched by calling the MetricService.Get RPC method, which streams all registered metrics to the caller in batches (server streaming RPC). Batching the returned metrics avoids requiring a large buffer or large RPC MTU.

The returned metric objects have flattened paths to the root. For example, the returned metrics (post detokenization and jsonified) might look something like:

{
  "/i2c1/failed_txns": 17,
  "/i2c1/total_txns": 2013,
  "/i2c1/gyro/resets": 24,
  "/i2c1/gyro/hangs": 1,
  "/spi1/thermocouple/reads": 242,
  "/spi1/thermocouple/temp_celsius": 34.52,
}

Note that there is no nesting of the groups; the nesting is implied from the path.

RPC service setup#

To expose a MetricService in your application, do the following:

Define metrics around the system, and put them in a group or list of metrics. Easy choices include for example the global_groups and global_metrics variables; or creat your own.
Create an instance of pw::metric::MetricService.
Register the service with your RPC server.

For example:

#include "pw_metric/global.h"
#include "pw_metric/metric.h"
#include "pw_metric/metric_service_nanopb.h"
#include "pw_rpc/server.h"

// Note: You must customize the RPC server setup; see pw_rpc.
Channel channels[] = {
    Channel::Create<1>(&uart_output),
};
Server server(channels);

// Metric service instance, pointing to the global metric objects.
// This could also point to custom per-product or application objects.
pw::metric::MetricService metric_service(pw::metric::global_metrics,
                                         pw::metric::global_groups);

void RegisterServices() {
  server.RegisterService(metric_service);
  // Register other services here.
}

void main() {
  // ... system initialization ...

  RegisterServices();

  // ... start your applcation ...
}

Attention

Take care when exporting metrics. Ensure appropriate access control is in place. In some cases it may make sense to entirely disable metrics export for production builds. Although reading metrics via RPC won’t influence the device, in some cases the metrics could expose sensitive information if product owners are not careful.

Attention

MetricService::Get is a synchronous RPC method

Calls to is MetricService::Get are blocking and will send all metrics immediately, even though it is a server-streaming RPC. This will work fine if the device doesn’t have too many metrics, or doesn’t have concurrent RPCs like logging, but could be a problem in some cases.

We plan to offer an async version where the application is responsible for pumping the metrics into the streaming response. This gives flow control to the application.

Size report#

The below size report shows the cost in code and memory for a few examples of metrics. This does not include the RPC service.

Label

Segment

Delta

1 metric and 1 group, no dump or export

+412

+28

(+) dump group and metrics to log

+968

+28

(+) 1 group (+) 4 metrics

+168

+64

Attention

At time of writing, the above sizes show an unexpectedly large flash impact. We are investigating why GCC is inserting large global static constructors per group, when all the logic should be reused across objects.

Metric Parser#

The metric_parser Python module provides functions to fetch and parse metrics from a device via RPC. It detokenizes the metric and group names and returns the data as a nested dictionary.

The module provides two functions corresponding to the server’s RPC methods:

get_all_metrics(): Uses the paginated MetricService.Walk RPC and handles all client-side pagination logic automatically. This is recommended for asynchronous transports and for fetching large metric sets.
parse_metrics(): Uses the server-streaming MetricService.Get RPC.

Design tradeoffs#

There are many possible approaches to metrics collection and aggregation. We’ve chosen some points on the tradeoff curve:

Atomic-sized metrics - Using simple metric objects with just uint32/float enables atomic operations. While it might be nice to support larger types, it is more useful to have safe metrics increment from interrupt subroutines.
No aggregate metrics (yet) - Aggregate metrics (e.g. average, max, min, histograms) are not supported, and must be built on top of the simple base metrics. By taking this route, we can considerably simplify the core metrics system and have aggregation logic in separate modules. Those modules can then feed into the metrics system - for example by creating multiple metrics for a single underlying metric. For example: “foo”, “foo_max”, “foo_min” and so on.

The other problem with automatic aggregation is that what period the aggregation happens over is often important, and it can be hard to design this cleanly into the API. Instead, this responsibility is pushed to the user who must take more care.

Note that we will add helpers for aggregated metrics.
No virtual metrics - An alternate approach to the concrete Metric class in the current module is to have a virtual interface for metrics, and then allow those metrics to have their own storage. This is attractive but can lead to many vtables and excess memory use in simple one-metric use cases.
Linked list registration - Using linked lists for registration is a tradeoff, accepting some memory overhead in exchange for flexibility. Other alternatives include a global table of metrics, which has the disadvantage of requiring centralizing the metrics – an impossibility for middleware like Pigweed.
Synchronization - The only synchronization guarantee provided by pw_metric is that increment and set are atomic. Other than that, users are on their own to synchonize metric collection and updating.
No fast metric lookup - The current design does not make it fast to lookup a metric at runtime; instead, one must run a linear search of the tree to find the matching metric. In most non-dynamic use cases, this is fine in practice, and saves having a more involved hash table. Metric updates will be through direct member or variable accesses.
Relying on C++ static initialization - In short, the convenience outweighs the cost and risk. Without static initializers, it would be impossible to automatically collect the metrics without post-processing the C++ code to find the metrics; a huge and debatably worthwhile approach. Metrics lists are constant initialized, so can be accessed safely during static initialization.
Both local & global support - Potentially just one approach (the local or global one) could be offered, making the module less complex. However, we feel the additional complexity is worthwhile since there are legimitate use cases for both e.g. PW_METRIC and PW_METRIC_GLOBAL. We’d prefer to have a well-tested upstream solution for these use cases rather than have customers re-implement one of these.

Roadmap & Status#

String metric names - pw_metric stores metric names as tokens. On one hand, this is great for production where having a compact binary is often a requirement to fit the application in the given part. However, in early development before flash is a constraint, string names are more convenient to work with since there is no need for host-side detokenization. We plan to add optional support for using supporting strings.
Aggregate metrics - We plan to add support for aggregate metrics on top of the simple metric mechanism, either as another module or as additional functionality inside this one. Likely examples include min/max,
Selectively enable or disable metrics - Currently the metrics are always enabled once included. In practice this is not ideal since many times only a few metrics are wanted in production, but having to strip all the metrics code is error prone. Instead, we will add support for controlling what metrics are enabled or disabled at compile time. This may rely on of C++20’s support for zero-sized members to fully remove the cost.
(Completed) Paginated RPC for Asynchronous Transports - The original server-streaming Get RPC is unsuitable for asynchronous transports where the server cannot guarantee transport readiness. This has been addressed by the introduction of the MetricService.Walk unary RPC, which uses a paginated, client-driven approach for reliable metric collection. This pattern also supports large metric sets that exceed the transport MTU.
Timer integration - We would like to add a stopwatch type mechanism to time multiple in-flight events.
C support - In practice it’s often useful or necessary to instrument C-only code. While it will be impossible to support the global registration system that the C++ version supports, we will figure out a solution to make instrumenting C code relatively smooth.
Global counter - We may add a global metric counter to help detect cases where post-initialization metrics manipulations are done.
Proto structure - It may be possible to directly map metrics to a custom proto structure, where instead of a name or token field, a tag field is provided. This could result in elegant export to an easily machine parsable and compact representation on the host. We may investigate this in the

future.
Safer data structures - At a cost of 4B per metric and 4B per group, it may be possible to make metric structure instantiation safe even in static constructors, and also make it safe to remove metrics dynamically. We will consider whether this tradeoff is the right one, since a 4B cost per metric is substantial on projects with many metrics.