pw_buf#

Contiguous buffer views with optional memory ownership

Unstable C++

pw_buf provides the pw::Buf and pw::ConstBuf classes.

Buf and ConstBuf#

pw::Buf and pw::ConstBuf are views into contiguous blocks of owned or unowned memory. The bytes in a Buf are mutable, while the bytes in a ConstBuf are read-only.

They can be interacted with like a std::span, but offer more functionality:

  • Automatic memory management: Owned memory is automatically freed back to its allocator when the Buf goes out of scope or is reset.

  • Slicing and reclaiming: A region backing a Buf can be sliced into a smaller region, creating a subspan view that can be passed along. These slices can be later reclaimed. This can be useful for reserving headers and footers, allowing someone else to populate the payload between.

Ownership and pw_allocator#

Buf regions may optionally be allocated via a pw_allocator, causing the region’s deallocator to travel with the Buf (and its slices) and automatically reclaim the memory when it is destroyed.

Despite its name, pw_allocator does not necessarily mean “heap allocation”. It simply provides an interface for requesting and releasing ownership of a block of memory. You can, for instance, create a pw_allocator implementation that wraps a single static buffer with an “in use” flag; a Buf created over it would just signal transfers of ownership without ever allocating any memory.

Slicing and reclaiming#

Both pw::Buf and pw::ConstBuf can be truncated or sliced. A Buf can reclaim previously truncated or sliced regions.

For example, say you are implementing a simple framing protocol consisting of a header followed by a payload. You would create a Buf large enough for both, if possible directly over the memory region used by the transport (e.g. a DMA buffer).

From this Buf, you can then slice the size of the header from the front, creating a new Buf that owns the full region, but only sees the payload. This sliced Buf can be handed up to a higher layer for it to populate.

Depending on the type of protocol, you could handle this in one of two ways:

  • If the packets have fixed size and parameters known up front, you can just pre-populate the header before slicing the payload Buf. Once the higher layer is done, they can immediately hand it back to the transport as a well-formed packet without further modifications.

  • Alternatively, if some header fields depend on the payload (e.g. length, checksum), you would reserve the header space upfront, hand the sliced Buf over, then have the higher layer return it to you. At that point, you would reclaim the prefix span, inspect the written payload, and write the header using the payload’s finalized state.

Since ownership of the full underlying region travels with each slice, the packet Buf can be safely destroyed at any point, returning its region back to the allocator that provided it.

This process can be repeated multiple layers up, creating a full protocol stack where each layer only knows about its own packet format, without ever copying data between layers.

The pw_buf API provides utility functions for trimming and restoring views of a buffer:

  • pw::Slice(): Shrinks a buffer view to a sub-range of its bytes.

  • pw::Truncate(): Truncates a buffer view to a smaller size from the start.

  • pw::Reclaim(): Expands a sliced buffer view back into its originally allocated prefix and suffix bytes.

Conversions#

A pw::Buf can be:

Examples#

Allocate and TryAllocate a Buf#

 1#include "pw_buf/buf.h"
 2
 3namespace examples {
 4
 5pw::Buf AllocateBuf(pw::Allocator& allocator) {
 6  // Allocate 100 bytes using the allocator.
 7  // This will PW_ASSERT if allocation fails.
 8  return pw::Buf::Allocate(allocator, 100);
 9}
10
11pw::Buf TryAllocateBuf(pw::Allocator& allocator) {
12  // Try to allocate 100 bytes. Returns an empty Buf on failure.
13  pw::Buf buf = pw::Buf::TryAllocate(allocator, 100);
14  if (buf.empty()) {
15    // Handle allocation failure.
16  }
17  return buf;
18}
19
20}  // namespace examples

Create a Buf from a UniquePtr#

 1#include "pw_buf/buf.h"
 2
 3namespace examples {
 4
 5pw::Buf CreateBufFromUniquePtr(pw::Allocator& allocator) {
 6  pw::UniquePtr<std::byte[]> unique_data =
 7      allocator.MakeUnique<std::byte[]>(100);
 8  if (unique_data == nullptr) {
 9    return pw::Buf();
10  }
11  // Construct a Buf by moving the UniquePtr into it.
12  // The Buf now owns the allocation.
13  return pw::Buf(std::move(unique_data));
14}
15
16}  // namespace examples

Pass a Buf as a std::span#

 1#include "pw_buf/buf.h"
 2
 3namespace examples {
 4
 5void ProcessData(pw::ByteSpan span) {
 6  if (!span.empty()) {
 7    span[0] = std::byte{0x42};
 8  }
 9}
10
11void PassBufAsSpan(pw::Buf& buf) {
12  // pw::Buf implicitly converts to pw::ByteSpan.
13  ProcessData(buf);
14}
15
16}  // namespace examples

Pass a Buf as a ConstBuf&#

 1#include "pw_buf/buf.h"
 2
 3namespace examples {
 4
 5void ReadData(const pw::ConstBuf& const_buf) {
 6  if (!const_buf.empty()) {
 7    // Read operations only
 8    std::byte first = const_buf[0];
 9    (void)first;
10  }
11}
12
13void PassBufAsConstBufReference(pw::Buf& buf) {
14  // pw::Buf implicitly converts to pw::ConstBuf&, allowing read-only access.
15  ReadData(buf);
16}
17
18}  // namespace examples

Slice and Reclaim a Buf#

 1#include "pw_buf/buf.h"
 2
 3namespace examples {
 4
 5pw::Buf SliceAndReclaim(pw::Buf&& buf) {
 6  // Trim 10 bytes from the front and 5 bytes from the back.
 7  // The resulting sliced buffer view has size = original_size - 15.
 8  pw::Buf sliced = pw::Slice(std::move(buf), 10, buf.size() - 15);
 9
10  // ... perform operations on the sliced buffer ...
11
12  // Reclaim the 10 prefix bytes and 5 suffix bytes.
13  return pw::Reclaim(std::move(sliced), 10, 5);
14}
15
16}  // namespace examples

Using Buf in a simple network stack#

The following snippet demonstrates how to use pw::Buf and pw::ConstBuf within a packet-oriented connection socket to implement a length-prefixed protocol.

  1/// An example socket that implements a simple 4-byte length-prefixed protocol.
  2class ExampleSocket : public Socket {
  3 public:
  4  explicit ExampleSocket(pw::Allocator& allocator)
  5      : allocator_(&allocator),
  6        read_queue_(allocator),
  7        outbound_queue_(allocator) {}
  8
  9  pw::ConstBuf Read() override {
 10    PW_CHECK(!read_queue_.empty());
 11    pw::ConstBuf front = std::move(read_queue_.front());
 12    read_queue_.pop_front();
 13    return front;
 14  }
 15
 16  bool HasReadPacket() const { return !read_queue_.empty(); }
 17
 18  WriteReservation ReserveWrite() override {
 19    // Allocate a buffer of 128 bytes. Reserve the first 4 bytes for the header.
 20    auto owned = allocator_->MakeUnique<std::byte[]>(128);
 21    pw::Buf buf(std::move(owned), sizeof(uint32_t), 128 - sizeof(uint32_t));
 22    return WriteReservation(std::move(buf), this);
 23  }
 24
 25  void EnqueueForRead(pw::UniquePtr<std::byte[]>&& owned) {
 26    if (owned == nullptr || owned.size() < sizeof(uint32_t)) {
 27      return;
 28    }
 29    uint32_t payload_len = 0;
 30    std::memcpy(&payload_len, owned.get(), sizeof(uint32_t));
 31    PW_CHECK(sizeof(uint32_t) + payload_len <= owned.size());
 32
 33    pw::ConstBuf payload_buf =
 34        pw::Buf(std::move(owned), sizeof(uint32_t), payload_len);
 35    read_queue_.push_back(std::move(payload_buf));
 36  }
 37
 38  bool HasOutboundPacket() const { return !outbound_queue_.empty(); }
 39
 40  pw::Buf PopOutboundPacket() {
 41    PW_CHECK(!outbound_queue_.empty());
 42    pw::Buf front = std::move(outbound_queue_.front());
 43    outbound_queue_.pop_front();
 44    return front;
 45  }
 46
 47  void Write(pw::Buf&& payload_buf) override {
 48    // Reclaim the 4-byte prefix space to write the length header.
 49    pw::Buf packet_buf =
 50        pw::ReclaimPrefix(std::move(payload_buf), sizeof(uint32_t));
 51
 52    // Write length header
 53    uint32_t payload_len =
 54        static_cast<uint32_t>(packet_buf.size() - sizeof(uint32_t));
 55    std::memcpy(packet_buf.data(), &payload_len, sizeof(uint32_t));
 56
 57    outbound_queue_.push_back(std::move(packet_buf));
 58  }
 59
 60 private:
 61  pw::Allocator* allocator_;
 62  pw::DynamicDeque<pw::ConstBuf> read_queue_;
 63  pw::DynamicDeque<pw::Buf> outbound_queue_;
 64};
 65
 66/// An example layered socket that wraps another socket and adds a 4-byte CRC
 67/// checksum footer, demonstrating zero-copy layered packet construction.
 68class ExampleLayeredSocket : public Socket {
 69 public:
 70  explicit ExampleLayeredSocket(Socket& lower_socket)
 71      : lower_socket_(&lower_socket) {}
 72
 73  pw::ConstBuf Read() override {
 74    pw::ConstBuf raw = lower_socket_->Read();
 75    PW_CHECK(raw.size() >= sizeof(uint32_t) * 2);
 76
 77    uint32_t payload_len = 0;
 78    std::memcpy(&payload_len, raw.data(), sizeof(uint32_t));
 79    PW_CHECK(sizeof(uint32_t) * 2 + payload_len <= raw.size());
 80
 81    uint32_t crc = 0;
 82    std::memcpy(
 83        &crc, raw.data() + sizeof(uint32_t) + payload_len, sizeof(uint32_t));
 84    PW_CHECK(crc == 0xDEADBEEF);
 85
 86    return pw::Slice(std::move(raw), sizeof(uint32_t), payload_len);
 87  }
 88
 89  void Write(pw::Buf&& payload_buf) override {
 90    // Reclaim 4 bytes at the front (for header) and 4 bytes at the end (for CRC
 91    // footer).
 92    pw::Buf packet_buf =
 93        pw::Reclaim(std::move(payload_buf), sizeof(uint32_t), sizeof(uint32_t));
 94
 95    // Write header (payload len)
 96    uint32_t payload_len =
 97        static_cast<uint32_t>(packet_buf.size() - sizeof(uint32_t) * 2);
 98    std::memcpy(packet_buf.data(), &payload_len, sizeof(uint32_t));
 99
100    // Write footer (CRC)
101    uint32_t crc = 0xDEADBEEF;
102    std::memcpy(packet_buf.data() + sizeof(uint32_t) + payload_len,
103                &crc,
104                sizeof(uint32_t));
105
106    lower_socket_->Write(std::move(packet_buf));
107  }
108
109  WriteReservation ReserveWrite() override {
110    // Delegate reservation to lower socket, then adopt it by shrinking
111    // the available payload space to leave room for the layered header/footer.
112    WriteReservation res = lower_socket_->ReserveWrite();
113    Adopt(res, sizeof(uint32_t), sizeof(uint32_t));
114    return res;
115  }
116
117 private:
118  Socket* lower_socket_;
119};