pw_multibuf#

A buffer API optimized for zero-copy messaging

Unstable C++17

Sending or receiving messages via RPC, transfer, or sockets often requires a series of intermediate buffers, each requiring their own copy of the data. pw_multibuf allows data to be written once, eliminating the memory, CPU and latency overhead of copying.

How does it work?#

pw_multibuf uses several techniques to minimize copying of data:

Header and Footer Reservation: Lower-level components can reserve space within a buffer for headers and/or footers. This allows headers and footers to be added to user-provided data without moving users’ data.
Native Scatter/Gather and Fragmentation Support: Buffers can refer to multiple separate chunks of memory. Messages can be built up from discontiguous allocations, and users’ data can be fragmented across multiple packets.
Divisible Memory Regions: Incoming buffers can be divided without a copy, allowing incoming data to be freely demultiplexed.

What kinds of data is this for?#

pw_multibuf is best used in code that wants to read, write, or pass along data which are one of the following:

Large: pw_multibuf is designed to allow breaking up data into multiple chunks. It also supports asynchronous allocation for when there may not be sufficient space for incoming data.
Communications-Oriented: Data which is being received or sent across sockets, various packets, or shared-memory protocols can benefit from the fragmentation, multiplexing, and header/footer-reservation properties of pw_multibuf.
Copy-Averse: pw_multibuf is structured to allow users to pass around and mutate buffers without copying or moving data in-memory. This can be especially useful when working in systems that are latency-sensitive, need to pass large amounts of data, or when memory usage is constrained.

API Reference#

Most users of pw_multibuf will start by allocating a MultiBuf using a MultiBufAllocator class.

MultiBuf s consist of a number of Chunk s of contiguous memory. These Chunk s can be grown, shrunk, modified, or extracted from the MultiBuf. MultiBuf exposes an std::byte iterator interface as well as a Chunk iterator available through the Chunks() method.

An RAII-style OwnedChunk is also provided, and manages the lifetime of Chunk s which are not currently stored inside of a MultiBuf.

class Chunk#

A handle to a contiguous slice of data.

A Chunk is similar to a ByteSpan, but is aware of the underlying memory allocation, and is able to split, shrink, and grow into neighboring empty space.

This class is optimized to allow multiple owners to write into neighboring regions of the same allocation. One important usecase for this is communication protocols which want to reserve space at the front or rear of a buffer for headers or footers.

In order to support zero-copy DMA of communications buffers, allocators can create properly-aligned Chunk regions in appropriate memory. The driver can then DiscardFront in order to reserve bytes for headers, Truncate in order to reserve bytes for footers, and then pass the Chunk to the user to fill in. The header and footer space can then be reclaimed using the ClaimPrefix and ClaimSuffix methods.

Public Functions

bool CanMerge(const Chunk &next_chunk) const#

Returns if next_chunk is mergeable into the end of this Chunk.

This will only succeed when the two Chunk s are adjacent in memory and originated from the same allocation.

bool Merge(OwnedChunk &next_chunk)#

Attempts to merge next_chunk into the end of this Chunk.

If the chunks are successfully merged, this Chunk will be extended forwards to encompass the space of next_chunk, and next_chunk will be emptied and Released.

This will only succeed when the two Chunk s are adjacent in memory and originated from the same allocation.

If the chunks are not mergeable, neither Chunk will be modified.

bool ClaimPrefix(size_t bytes_to_claim)#

Attempts to add bytes_to_claim to the front of this buffer by advancing its range backwards in memory. Returns true if the operation succeeded.

This will only succeed if this Chunk points to a section of a region that has unreferenced bytes preceeding it. For example, a Chunk which has been shrunk using DiscardFront can be re-expanded using ClaimPrefix.

This method will acquire a mutex and is not IRQ safe.

bool ClaimSuffix(size_t bytes_to_claim)#

Attempts to add bytes_to_claim to the front of this buffer by advancing its range forwards in memory. Returns true if the operation succeeded.

This will only succeed if this Chunk points to a section of a region that has unreferenced bytes following it. For example, a Chunk which has been shrunk using Truncate can be re-expanded using

This method will acquire a mutex and is not IRQ safe. /// ClaimSuffix.

void DiscardFront(size_t bytes_to_discard)#

Shrinks this handle to refer to the data beginning at offset bytes_to_discard.

Does not modify the underlying data.

This method will acquire a mutex and is not IRQ safe.

void Slice(size_t begin, size_t end)#

Shrinks this handle to refer to data in the range begin..<end.

Does not modify the underlying data.

This method will acquire a mutex and is not IRQ safe.

void Truncate(size_t len)#

Shrinks this handle to refer to only the first len bytes.

Does not modify the underlying data.

This method will acquire a mutex and is not IRQ safe.

std::optional<OwnedChunk> TakeFront(size_t bytes_to_take)#

Attempts to shrink this handle to refer to the data beginning at offset bytes_to_take, returning the first bytes_to_take bytes as a new OwnedChunk.

If the inner call to AllocateChunkClass fails, this function will return `std::nullopt and this handle’s span will not change.

This method will acquire a mutex and is not IRQ safe.

std::optional<OwnedChunk> TakeTail(size_t bytes_to_take)#

Attempts to shrink this handle to refer only the first len - bytes_to_take bytes, returning the last bytes_to_take bytes as a new OwnedChunk.

If the inner call to AllocateChunkClass fails, this function will return std::nullopt and this handle’s span will not change.

This method will acquire a mutex and is not IRQ safe.

Public Static Functions

static std::optional<OwnedChunk> CreateFirstForRegion(ChunkRegionTracker &region_tracker)#

Creates the first Chunk referencing a whole region of memory.

This must only be called once per ChunkRegionTracker, when the region is first created. Multiple calls will result in undefined behavior.

Returns std::nullopt if AllocateChunkStorage returns nullptr.

class OwnedChunk#

An RAII handle to a contiguous slice of data.

Note: OwnedChunk may acquire a pw::sync::Mutex during destruction, and so must not be destroyed within ISR contexts.

Public Functions

inline ~OwnedChunk()#: This method will acquire a mutex and is not IRQ safe.

void Release()#

Decrements the reference count on the underlying chunk of data and empties this handle so that span() now returns an empty (zero-sized) span.

Does not modify the underlying data, but may cause it to be deallocated if this was the only remaining Chunk referring to its region.

This method is equivalent to { Chunk _unused = std::move(chunk_ref); }

This method will acquire a mutex and is not IRQ safe.

inline Chunk *Take() &&#: Returns the contained Chunk* and empties this OwnedChunk without releasing the underlying Chunk.

class MultiBuf#

A buffer optimized for zero-copy data transfer.

A MultiBuf consists of multiple Chunk s of data.

Public Functions

void Release() noexcept#

Decrements the reference count on the underlying chunks of data and empties this MultiBuf so that size() == 0.

Does not modify the underlying data, but may cause it to be deallocated

This method is equivalent to { MultiBuf _unused = std::move(multibuf); }

This method will acquire a mutex and is not IRQ safe.

inline ~MultiBuf()#: This destructor will acquire a mutex and is not IRQ safe.

size_t size() const#

Returns the number of bytes in this container.

This method’s complexity is O(Chunks().size()).

void PushFrontChunk(OwnedChunk chunk)#

Pushes Chunk onto the front of the MultiBuf.

This operation does not move any data and is O(1).

ChunkIterator InsertChunk(ChunkIterator position, OwnedChunk chunk)#

Inserts chunk into the specified position in the MultiBuf.

This operation does not move any data and is O(Chunks().size()).

Returns an iterator pointing to the newly-inserted Chunk.

std::tuple<ChunkIterator, OwnedChunk> TakeChunk(ChunkIterator position)#

Removes a Chunk from the specified position.

This operation does not move any data and is O(Chunks().size()).

Returns an iterator pointing to the Chunk after the removed Chunk, or Chunks().end() if this was the last Chunk in the MultiBuf.

inline iterator begin()#: Returns an iterator pointing to the first byte of this `MultiBuf.

inline const_iterator begin() const#: Returns a const iterator pointing to the first byte of this `MultiBuf.

inline const_iterator cbegin() const#: Returns a const iterator pointing to the first byte of this `MultiBuf.

inline iterator end()#: Returns an iterator pointing to the end of this MultiBuf.

inline const_iterator end() const#: Returns a const iterator pointing to the end of this MultiBuf.

inline const_iterator cend() const#: Returns a const iterator pointing to the end of this MultiBuf.

inline constexpr ChunkIterable Chunks()#: Returns an iterable container which yields the Chunks in this MultiBuf.

inline constexpr const ChunkIterable Chunks() const#: Returns an iterable container which yields the const Chunks in this MultiBuf.

inline constexpr ChunkIterator ChunkBegin()#: Returns an iterator pointing to the first Chunk in this MultiBuf.

inline constexpr ChunkIterator ChunkEnd()#: Returns an iterator pointing to the end of the Chunks in this MultiBuf.

inline constexpr ConstChunkIterator ConstChunkBegin()#: Returns a const iterator pointing to the first Chunk in this MultiBuf.

inline constexpr ConstChunkIterator ConstChunkEnd()#: Returns a const iterator pointing to the end of the Chunks in this MultiBuf.

class ChunkIterable#

An iterable containing the Chunk s of a MultiBuf.

Public Functions

size_t size() const#: Returns the number of Chunks in this iterable.

class ChunkIterator#: A std::forward_iterator over the Chunks of a MultiBuf.

class const_iterator#

A const std::forward_iterator over the bytes of a MultiBuf.

Public Functions

inline constexpr const Chunk *chunk() const#: Returns the current Chunk pointed to by this iterator.

inline constexpr size_t byte_index() const#: Returns the index of the byte pointed to by this iterator within the current Chunk.

class ConstChunkIterator#: A const std::forward_iterator over the Chunks of a MultiBuf.

class iterator#

An std::forward_iterator over the bytes of a MultiBuf.

Public Functions

inline constexpr Chunk *chunk() const#: Returns the current Chunk pointed to by this iterator.

inline constexpr size_t byte_index() const#: Returns the index of the byte pointed to by this iterator within the current Chunk.

class MultiBufAllocator#

Allocator Implementors’ API#

Some users will need to directly implement the MultiBufAllocator interface in order to provide allocation out of a particular region, provide particular allocation policy, fix Chunks to some size (such as MTU size - header for socket implementations), or specify other custom behavior.

These users will also need to understand and implement the following APIs:

class ChunkRegionTracker#

An object that manages a single allocated region which is referenced by one or more Chunk objects.

This class is typically implemented by MultiBufAllocator implementations in order to customize the behavior of region deallocation.

ChunkRegionTracker s have three responsibilities:

Tracking the region of memory into which Chunk s can expand. This is reported via the Region method. Chunk s in this region can refer to memory within this region sparsely, but they can grow or shrink so long as they stay within the bounds of the Region.
Deallocating the region and the ChunkRegionTracker itself. This is implemented via the Destroy method, which is called once all of the Chunk s in a region have been released.
Allocating and deallocating space for Chunk classes. This is merely allocating space for the Chunk object itself, not the memory to which it refers. This can be implemented straightforwardly by delegating to an existing generic allocator such as malloc or a pw::allocator::Allocator implementation.

Subclassed by pw::multibuf::HeaderChunkRegionTracker

A simple implementation of a ChunkRegionTracker is provided, called HeaderChunkRegionTracker. It stores its Chunk and region metadata in a Allocator allocation alongside the data. The allocation process is synchronous, making this class suitable for testing. The allocated region or Chunk must not outlive the provided allocator.

class HeaderChunkRegionTracker : public pw::multibuf::ChunkRegionTracker #

A ChunkRegionTracker which stores its Chunk and region metadata in a allocator::Allocator allocation alongside the data.

This is useful when testing and when there is no need for asynchronous allocation.

Public Functions

inline virtual ByteSpan Region() const final#

Returns the entire span of the region being managed.

Chunk s referencing this tracker will not expand beyond this region, nor into one another’s portions of the region.

This region must not change for the lifetime of this ChunkRegionTracker.

Public Static Functions

static inline std::optional<OwnedChunk> AllocateRegionAsChunk(allocator::Allocator *alloc, size_t size)#

Allocates a new Chunk region of size bytes in alloc.

The underlyiing allocation will also store the HeaderChunkRegionTracker itself. The allocated memory must not outlive the provided allocator alloc.

Returns the newly-created OwnedChunk if successful.

static inline HeaderChunkRegionTracker *AllocateRegion(allocator::Allocator *alloc, size_t size)#

Allocates a new region of size bytes in alloc.

The underlyiing allocation will also store the HeaderChunkRegionTracker itself. The allocated memory must not outlive the provided allocator alloc.

Returns a pointer to the newly-created HeaderChunkRegionTracker or nullptr if the allocation failed.