pw_buf#

Contiguous buffer views with optional memory ownership

Unstable C++

pw_buf provides the pw::Buf and pw::ConstBuf classes.

Buf and ConstBuf#

pw::Buf and pw::ConstBuf are views into contiguous blocks of owned or unowned memory. The bytes in a Buf are mutable, while the bytes in a ConstBuf are read-only.

They can be interacted with like a std::span, but offer more functionality:

  • Automatic memory management: Owned memory is automatically freed back to its allocator when the Buf goes out of scope or is reset.

  • Slicing and reclaiming: A region backing a Buf can be sliced into a smaller region, creating a subspan view that can be passed along. These slices can be later reclaimed. This can be useful for reserving headers and footers, allowing someone else to populate the payload between.

Ownership and pw_allocator#

Buf regions may optionally be allocated via a pw_allocator, causing the region’s deallocator to travel with the Buf (and its slices) and automatically reclaim the memory when it is destroyed.

Despite its name, pw_allocator does not necessarily mean “heap allocation”. It simply provides an interface for requesting and releasing ownership of a block of memory. You can, for instance, create a pw_allocator implementation that wraps a single static buffer with an “in use” flag; a Buf created over it would just signal transfers of ownership without ever allocating any memory.

Slicing and reclaiming#

Both pw::Buf and pw::ConstBuf can be truncated or sliced. A Buf can reclaim previously truncated or sliced regions.

For example, say you are implementing a simple framing protocol consisting of a header followed by a payload. You would create a Buf large enough for both, if possible directly over the memory region used by the transport (e.g. a DMA buffer).

From this Buf, you can then slice the size of the header from the front, creating a new Buf that owns the full region, but only sees the payload. This sliced Buf can be handed up to a higher layer for it to populate.

Depending on the type of protocol, you could handle this in one of two ways:

  • If the packets have fixed size and parameters known up front, you can just pre-populate the header before slicing the payload Buf. Once the higher layer is done, they can immediately hand it back to the transport as a well-formed packet without further modifications.

  • Alternatively, if some header fields depend on the payload (e.g. length, checksum), you would reserve the header space upfront, hand the sliced Buf over, then have the higher layer return it to you. At that point, you would reclaim the prefix span, inspect the written payload, and write the header using the payload’s finalized state.

Since ownership of the full underlying region travels with each slice, the packet Buf can be safely destroyed at any point, returning its region back to the allocator that provided it.

This process can be repeated multiple layers up, creating a full protocol stack where each layer only knows about its own packet format, without ever copying data between layers.

The pw_buf API provides utility functions for trimming and restoring views of a buffer:

  • pw::Slice(): Shrinks a buffer view to a sub-range of its bytes.

  • pw::Truncate(): Truncates a buffer view to a smaller size from the start.

  • pw::Reclaim(): Expands a sliced buffer view back into its originally allocated prefix and suffix bytes.

Relationship between Buf and ConstBuf#

pw::Buf and pw::ConstBuf manage owned or unowned buffer memory, but serve different purposes:

  • Mutable vs. read-only: A Buf provides mutable access to its bytes, whereas a ConstBuf is strictly read-only.

  • Slicing and reclaiming: A Buf can be sliced (pw::Slice()) and later reclaimed (pw::Reclaim()), making it ideal for constructing write packets where headers or footers are reserved and populated in stages. A ConstBuf only supports slicing and truncating, never reclaiming, which is suited for read packets where layers strip headers as data moves up the protocol stack without accessing outside their assigned slice.

A Buf can be converted to a ConstBuf by moving it:

  pw::Buf buf = examples::AllocateBuf(allocator);
  pw::ConstBuf const_buf = std::move(buf);

Moving the Buf transfers ownership and leaves the Buf null. A Buf may also be used through a const ConstBuf&, though accepting a pw::ConstByteSpan (or pw::span<const std::byte>) by value is recommended for functions that only borrow data for reading.

Relationship to pw_multibuf#

Conceptually, a pw_multibuf is a sequence of layered pw::Buf or pw::ConstBuf instances. Both modules provide abstractions for zero-copy buffer management and safe ownership tracking with pw_allocator, operating at different levels of granularity:

  • Contiguous buffer vs. sequence of buffers: pw::Buf and pw::ConstBuf manage a single, contiguous block of memory. In contrast, a MultiBuf aggregates a sequence of multiple contiguous buffers (chunks) into a unified virtual span.

  • Slicing vs. layering: A Buf allows in-place slicing and reclamation of reserved prefix/suffix bytes within its single allocation. A MultiBuf generalizes this across its sequence of buffers, allowing callers to add and pop non-destructive layers over the entire collection of chunks.

  • Overhead: pw::Buf has minimal metadata overhead (storing only view pointers and an optional deallocator), making it lightweight for operations on a single buffer. pw_multibuf maintains an internal deque of chunk and layer metadata to manage arbitrary fragments and layers across the sequence.

pw_buf and pw_multibuf integrate directly:

  • Ingestion: A pw::Buf or pw::ConstBuf can be inserted or appended into a MultiBuf via Insert() or PushBack(), transferring ownership of the underlying allocation into the MultiBuf.

  • Extraction: Contiguous owned memory can be extracted from a MultiBuf back into a pw::Buf or pw::ConstBuf using Release(). Sliced views are preserved so that only the visible bytes are exposed, while the full underlying allocation is safely tracked for deallocation.

When to use which:

  • Use pw_buf when working with a single contiguous allocation (e.g. DMA buffers, fixed packet buffers), especially when reserving and populating headers and footers in-place via pw::Slice() and pw::Reclaim().

  • Use pw_multibuf when assembling or disassembling data across multiple disjoint buffers, performing scatter-gather I/O, or composing multi-layer network packets from disparate memory sources.

Examples#

Allocate and TryAllocate a Buf#

 1#include "pw_buf/buf.h"
 2
 3namespace examples {
 4
 5pw::Buf AllocateBuf(pw::Allocator& allocator) {
 6  // Allocate 100 bytes using the allocator.
 7  // This will PW_ASSERT if allocation fails.
 8  return pw::Buf::Allocate(allocator, 100);
 9}
10
11pw::Buf TryAllocateBuf(pw::Allocator& allocator) {
12  // Try to allocate 100 bytes. Returns a null Buf on failure.
13  pw::Buf buf = pw::Buf::TryAllocate(allocator, 100);
14  if (buf == nullptr) {
15    // Handle allocation failure.
16  }
17  return buf;
18}
19
20}  // namespace examples

Create a Buf from a UniquePtr#

 1#include "pw_buf/buf.h"
 2
 3namespace examples {
 4
 5pw::Buf CreateBufFromUniquePtr(pw::Allocator& allocator) {
 6  pw::UniquePtr<std::byte[]> unique_data =
 7      allocator.MakeUnique<std::byte[]>(100);
 8  if (unique_data == nullptr) {
 9    return pw::Buf();
10  }
11  // Construct a Buf by moving the UniquePtr into it.
12  // The Buf now owns the allocation.
13  return pw::Buf(std::move(unique_data));
14}
15
16}  // namespace examples

Pass a Buf as a std::span#

 1#include "pw_buf/buf.h"
 2
 3namespace examples {
 4
 5void ProcessData(pw::ByteSpan span) {
 6  if (!span.empty()) {
 7    span[0] = std::byte{0x42};
 8  }
 9}
10
11void PassBufAsSpan(pw::Buf& buf) {
12  // pw::Buf implicitly converts to pw::ByteSpan.
13  ProcessData(buf);
14}
15
16}  // namespace examples

Slice and Reclaim a Buf#

 1#include "pw_buf/buf.h"
 2
 3namespace examples {
 4
 5pw::Buf SliceAndReclaim(pw::Buf&& buf) {
 6  // Trim 10 bytes from the front and 5 bytes from the back.
 7  // The resulting sliced buffer view has size = original_size - 15.
 8  pw::Buf sliced = pw::Slice(std::move(buf), 10, buf.size() - 15);
 9
10  // ... perform operations on the sliced buffer ...
11
12  // Reclaim the 10 prefix bytes and 5 suffix bytes.
13  return pw::Reclaim(std::move(sliced), 10, 5);
14}
15
16}  // namespace examples

Using Buf in a simple network stack#

The following snippet demonstrates how to use pw::Buf and pw::ConstBuf within a packet-oriented connection socket to implement a length-prefixed protocol.

  1/// An example socket that implements a simple 4-byte length-prefixed protocol.
  2class ExampleSocket : public Socket {
  3 public:
  4  explicit ExampleSocket(pw::Allocator& allocator)
  5      : allocator_(&allocator),
  6        read_queue_(allocator),
  7        outbound_queue_(allocator) {}
  8
  9  pw::ConstBuf Read() override {
 10    PW_CHECK(!read_queue_.empty());
 11    pw::ConstBuf front = std::move(read_queue_.front());
 12    read_queue_.pop_front();
 13    return front;
 14  }
 15
 16  bool HasReadPacket() const { return !read_queue_.empty(); }
 17
 18  WriteReservation ReserveWrite() override {
 19    // Allocate a buffer of 128 bytes. Reserve the first 4 bytes for the header.
 20    auto owned = allocator_->MakeUnique<std::byte[]>(128);
 21    pw::Buf buf(std::move(owned), sizeof(uint32_t), 128 - sizeof(uint32_t));
 22    return WriteReservation(std::move(buf), this);
 23  }
 24
 25  void EnqueueForRead(pw::UniquePtr<std::byte[]>&& owned) {
 26    if (owned == nullptr || owned.size() < sizeof(uint32_t)) {
 27      return;
 28    }
 29    uint32_t payload_len = 0;
 30    std::memcpy(&payload_len, owned.get(), sizeof(uint32_t));
 31    PW_CHECK(sizeof(uint32_t) + payload_len <= owned.size());
 32
 33    pw::ConstBuf payload_buf =
 34        pw::Buf(std::move(owned), sizeof(uint32_t), payload_len);
 35    read_queue_.push_back(std::move(payload_buf));
 36  }
 37
 38  bool HasOutboundPacket() const { return !outbound_queue_.empty(); }
 39
 40  pw::Buf PopOutboundPacket() {
 41    PW_CHECK(!outbound_queue_.empty());
 42    pw::Buf front = std::move(outbound_queue_.front());
 43    outbound_queue_.pop_front();
 44    return front;
 45  }
 46
 47  void Write(pw::Buf&& payload_buf) override {
 48    // Reclaim the 4-byte prefix space to write the length header.
 49    pw::Buf packet_buf =
 50        pw::ReclaimPrefix(std::move(payload_buf), sizeof(uint32_t));
 51
 52    // Write length header
 53    uint32_t payload_len =
 54        static_cast<uint32_t>(packet_buf.size() - sizeof(uint32_t));
 55    std::memcpy(packet_buf.data(), &payload_len, sizeof(uint32_t));
 56
 57    outbound_queue_.push_back(std::move(packet_buf));
 58  }
 59
 60 private:
 61  pw::Allocator* allocator_;
 62  pw::DynamicDeque<pw::ConstBuf> read_queue_;
 63  pw::DynamicDeque<pw::Buf> outbound_queue_;
 64};
 65
 66/// An example layered socket that wraps another socket and adds a 4-byte CRC
 67/// checksum footer, demonstrating zero-copy layered packet construction.
 68class ExampleLayeredSocket : public Socket {
 69 public:
 70  explicit ExampleLayeredSocket(Socket& lower_socket)
 71      : lower_socket_(&lower_socket) {}
 72
 73  pw::ConstBuf Read() override {
 74    pw::ConstBuf raw = lower_socket_->Read();
 75    PW_CHECK(raw.size() >= sizeof(uint32_t) * 2);
 76
 77    uint32_t payload_len = 0;
 78    std::memcpy(&payload_len, raw.data(), sizeof(uint32_t));
 79    PW_CHECK(sizeof(uint32_t) * 2 + payload_len <= raw.size());
 80
 81    uint32_t crc = 0;
 82    std::memcpy(
 83        &crc, raw.data() + sizeof(uint32_t) + payload_len, sizeof(uint32_t));
 84    PW_CHECK(crc == 0xDEADBEEF);
 85
 86    return pw::Slice(std::move(raw), sizeof(uint32_t), payload_len);
 87  }
 88
 89  void Write(pw::Buf&& payload_buf) override {
 90    // Reclaim 4 bytes at the front (for header) and 4 bytes at the end (for CRC
 91    // footer).
 92    pw::Buf packet_buf =
 93        pw::Reclaim(std::move(payload_buf), sizeof(uint32_t), sizeof(uint32_t));
 94
 95    // Write header (payload len)
 96    uint32_t payload_len =
 97        static_cast<uint32_t>(packet_buf.size() - sizeof(uint32_t) * 2);
 98    std::memcpy(packet_buf.data(), &payload_len, sizeof(uint32_t));
 99
100    // Write footer (CRC)
101    uint32_t crc = 0xDEADBEEF;
102    std::memcpy(packet_buf.data() + sizeof(uint32_t) + payload_len,
103                &crc,
104                sizeof(uint32_t));
105
106    lower_socket_->Write(std::move(packet_buf));
107  }
108
109  WriteReservation ReserveWrite() override {
110    // Delegate reservation to lower socket, then adopt it by shrinking
111    // the available payload space to leave room for the layered header/footer.
112    WriteReservation res = lower_socket_->ReserveWrite();
113    Adopt(res, sizeof(uint32_t), sizeof(uint32_t));
114    return res;
115  }
116
117 private:
118  Socket* lower_socket_;
119};