pw_log_rpc

An RPC-based logging backend for Pigweed.

Warning

This module is under construction and might change in the future.

How to use

1. Set up RPC

Set up RPC for your target device. Basic deployments run RPC over a UART, with HDLC on top for framing. See pw_rpc for details on how to enable pw_rpc.

2. Set up tokenized logging (optional)

Set up the pw_log_tokenized log backend.

3. Connect the tokenized logging handler to the MultiSink

Create a MultiSink instance to buffer log entries. Then, make the log backend handler, pw_tokenizer_HandleEncodedMessageWithPayload, encode log entries in the log::LogEntry format, and add them to the MultiSink.

4. Create log drains

Create an RpcLogDrainMap with one RpcLogDrain for each RPC channel used to stream logs. Provide this map to the LogService and register the latter with the application’s RPC service. The RpcLogDrainMap provides a convenient way to access and maintain each RpcLogDrain. Attach each RpcLogDrain to the MultiSink.

5. Flush the log drains in the background

Depending on the product’s requirements, create a thread to flush all RpcLogDrains or one thread per drain. The thread(s) must continuously call RpcLogDrain::Flush() to pull entries from the MultiSink and send them to the log listeners.

Logging over RPC diagrams

Sample RPC logs request

The log listener, e.g. a computer, requests logs via RPC. The log service receives the request and sets up the corresponding RpcLogDrain to start the log stream.

graph TD computer[Computer]-->pw_rpc; pw_rpc-->log_service[LogService]; log_service-->rpc_log_drain_pc[RpcLogDrain<br>streams to<br>computer];;

Sample logging over RPC

Logs are streamed via RPC to a computer, and to another log listener. There can also be internal log readers, i.e. MultiSink::Drains, attached to the MultiSink, such as a writer to persistent memory, for example.

graph TD source1[Source 1]-->log_api[pw_log API]; source2[Source 2]-->log_api; log_api-->log_backend[Log backend]; log_backend-->multisink[MultiSink]; multisink-->drain[MultiSink::Drain]; multisink-->rpc_log_drain_pc[RpcLogDrain<br>streams to<br>computer]; multisink-->rpc_log_drain_other[RpcLogDrain<br>streams to<br>other log listener]; drain-->other_consumer[Other log consumer<br>e.g. persistent memory]; rpc_log_drain_pc-->pw_rpc; rpc_log_drain_other-->pw_rpc; pw_rpc-->computer[Computer]; pw_rpc-->other_listener[Other log<br>listener];

RPC log service

The LogService class is an RPC service that provides a way to request a log stream sent via RPC. Thus, it helps avoid using a different protocol for logs and RPCs over the same interface(s). It requires a map of RpcLogDrains to assign stream writers and delegate the log stream flushing to the user’s preferred method.

RpcLogDrain

An RpcLogDrain reads from the MultiSink instance that buffers logs, then packs, and sends the retrieved log entries to the log listener. One RpcLogDrain is needed for each log listener. An RpcLogDrain needs a thread to continuously call Flush() to maintain the log stream. A thread can maintain multiple log streams, but it must not be the same thread used by the RPC server, to avoid blocking it.

Each RpcLogDrain is identified by a known RPC channel ID and requires a rpc::RawServerWriter to write the packed multiple log entries. This writer is assigned by the LogService::Listen RPC. Future work will allow RpcLogDrains to have an open RPC writer, to constantly stream logs without the need to request them. This is useful in cases where the connection to the client is dropped silently because the log stream can continue when reconnected without the client requesting it.

An RpcLogDrain must be attached to a MultiSink containing multiple log::LogEntrys. When Flush is called, the drain acquires the rpc::RawServerWriter ‘s write buffer, grabs one log::LogEntry from the multisink, encodes it into a log::LogEntries stream, and repeats the process until the write buffer is full. Then the drain calls rpc::RawServerWriter::Write to flush the write buffer and repeats the process until all the entries in the MultiSink are read or an error is found.

The user must provide a buffer large enough for the largest entry in the MultiSink while also accounting for the interface’s Maximum Transmission Unit (MTU). If the RpcLogDrain finds a drop message count as it reads the MultiSink it will insert a message in the stream with the drop message count.

RpcLogDrainMap

Provides a convenient way to access all or a single RpcLogDrain by its RPC channel ID.

RpcLogDrainThread

The module includes a sample thread that flushes each drain sequentially. Future work might replace this with enqueueing the flush work on a work queue. The user can also choose to have different threads flushing individual RpcLogDrains with different priorities.

Logging example

The following code shows a sample setup to defer the log handling to the RpcLogDrainThread to avoid having the log streaming block at the log callsite.

main.cc

#include "foo/foo_log.h"
#include "pw_log/log.h"
#include "pw_thread/detached_thread.h"
#include "pw_thread_stl/options.h"

namespace {

void RegisterServices() {
  pw::rpc::system_server::Server().RegisterService(foo_log::log_service);
}
}  // namespace

int main() {
  PW_LOG_INFO("Deferred logging over RPC example");
  pw::rpc::system_server::Init();
  RegisterServices();
  pw::thread::DetachedThread(pw::thread::stl::Options(), foo_log::log_thread);
  pw::rpc::system_server::Start();
  return 0;
}

foo_log.cc

Example of a log backend implementation, where logs enter the MultiSink and log drains are set up.

#include "foo/foo_log.h"

#include <array>
#include <cstdint>

#include "pw_chrono/system_clock.h"
#include "pw_log/proto_utils.h"
#include "pw_log_rpc/log_service.h"
#include "pw_log_rpc/rpc_log_drain.h"
#include "pw_log_rpc/rpc_log_drain_map.h"
#include "pw_log_rpc/rpc_log_drain_thread.h"
#include "pw_rpc_system_server/rpc_server.h"
#include "pw_sync/interrupt_spin_lock.h"
#include "pw_sync/lock_annotations.h"
#include "pw_sync/mutex.h"
#include "pw_tokenizer/tokenize_to_global_handler_with_payload.h"

namespace foo_log {
namespace {
constexpr size_t kLogBufferSize = 5000;
// Tokenized logs are typically 12-24 bytes.
constexpr size_t kMaxMessageSize = 32;
// kMaxLogEntrySize should be less than the MTU of the RPC channel output used
// by the provided server writer.
constexpr size_t kMaxLogEntrySize =
    pw::log_rpc::RpcLogDrain::kMinEntrySizeWithoutPayload + kMaxMessageSize;
std::array<std::byte, kLogBufferSize> multisink_buffer;

// To save RAM, share the mutex, since drains will be managed sequentially.
pw::sync::Mutex shared_mutex;
std::array<std::byte, kMaxEntrySize> client1_buffer
    PW_GUARDED_BY(shared_mutex);
std::array<std::byte, kMaxEntrySize> client2_buffer
    PW_GUARDED_BY(shared_mutex);
std::array<pw::log_rpc::RpcLogDrain, 2> drains = {
    pw::log_rpc::RpcLogDrain(
        1,
        client1_buffer,
        shared_mutex,
        RpcLogDrain::LogDrainErrorHandling::kIgnoreWriterErrors),
    pw::log_rpc::RpcLogDrain(
        2,
        client2_buffer,
        shared_mutex,
        RpcLogDrain::LogDrainErrorHandling::kIgnoreWriterErrors),
};

pw::sync::InterruptSpinLock log_encode_lock;
std::array<std::byte, kMaxLogEntrySize> log_encode_buffer
    PW_GUARDED_BY(log_encode_lock);

extern "C" void pw_tokenizer_HandleEncodedMessageWithPayload(
    pw_tokenizer_Payload metadata, const uint8_t message[], size_t size_bytes) {
  int64_t timestamp =
      pw::chrono::SystemClock::now().time_since_epoch().count();
  std::lock_guard lock(log_encode_lock);
  pw::Result<pw::ConstByteSpan> encoded_log_result =
    pw::log::EncodeTokenizedLog(
        metadata, message, size_bytes, timestamp, log_encode_buffer);

  if (!encoded_log_result.ok()) {
    GetMultiSink().HandleDropped();
    return;
  }
  GetMultiSink().HandleEntry(encoded_log_result.value());
}
}  // namespace

pw::log_rpc::RpcLogDrainMap drain_map(drains);
pw::log_rpc::RpcLogDrainThread log_thread(GetMultiSink(), drain_map);
pw::log_rpc::LogService log_service(drain_map);

pw::multisink::MultiSink& GetMultiSink() {
  static pw::multisink::MultiSink multisink(multisink_buffer);
  return multisink;
}
}  // namespace foo_log

Logging in other source files

To defer logging, other source files must simply include pw_log/log.h and use the pw_log APIs, as long as the source set that includes foo_log.cc is setup as the log backend.