pw_varint

Oveview

Functions for encoding and decoding variable length integers. Main docs: https://pigweed.dev/pw_varint.

Variable-length integer encoding and decoding library

Enumerations
enum class	pw::varint::Format { kZeroTerminatedLeastSignificant = PW_VARINT_ZERO_TERMINATED_LEAST_SIGNIFICANT , kZeroTerminatedMostSignificant = PW_VARINT_ZERO_TERMINATED_MOST_SIGNIFICANT , kOneTerminatedLeastSignificant = PW_VARINT_ONE_TERMINATED_LEAST_SIGNIFICANT , kOneTerminatedMostSignificant = PW_VARINT_ONE_TERMINATED_MOST_SIGNIFICANT }
	Describes a custom varint format.

Functions
StatusWithSize	pw::varint::Read (stream::Reader &reader, int64_t *output, size_t max_size=std::numeric_limits< size_t >::max())
	Decodes a variable-length integer (varint) from the current position of a pw::stream. Reads a maximum of 10 bytes or max_size, whichever is smaller.
StatusWithSize	pw::varint::Read (stream::Reader &reader, uint64_t *output, size_t max_size=std::numeric_limits< size_t >::max())
template<typename T >
constexpr std::make_unsigned_t< T >	pw::varint::ZigZagEncode (T n)
template<typename T >
constexpr std::make_signed_t< T >	pw::varint::ZigZagDecode (T n)
template<typename T , typename = std::enable_if_t<std::is_integral<T>::value || std::is_convertible<T, uint64_t>::value>>
constexpr size_t	pw::varint::EncodedSize (T integer)
	Computes the size of an integer when encoded as a varint.
size_t	pw::varint::EncodeLittleEndianBase128 (uint64_t integer, const span< std::byte > &output)
template<typename T >
size_t	pw::varint::Encode (T integer, const span< std::byte > &output)
size_t	pw::varint::Decode (const span< const std::byte > &input, int64_t *output)
size_t	pw::varint::Decode (const span< const std::byte > &input, uint64_t *value)
	This is an overloaded member function, provided for convenience. It differs from the above function only in what argument(s) it accepts.
size_t	pw::varint::Encode (uint64_t value, span< std::byte > output, Format format)
	Encodes a varint in a custom format.
size_t	pw::varint::Decode (span< const std::byte > input, uint64_t *value, Format format)
	Decodes a varint from a custom format.
constexpr uint64_t	pw::varint::MaxValueInBytes (size_t bytes)
	Returns the maximum (max) integer value that can be encoded as a varint into the specified number of bytes.

Variables
constexpr size_t	pw::varint::kMaxVarint32SizeBytes = 5
	Maximum size of a varint (LEB128) encoded uint32_t.
constexpr size_t	pw::varint::kMaxVarint64SizeBytes = 10
	Maximum size of a varint (LEB128) encoded uint64_t.

Function Documentation

Decode()

size_t pw::varint::Decode ( const span< const std::byte > &  input, int64_t *  output  )

inline

Decodes a varint-encoded value. If reading into a signed integer, the value is ZigZag decoded.

Returns the number of bytes read from the input if successful. Returns zero if the result does not fit in a int64_t/ uint64_t or if the input is exhausted before the number terminates. Reads a maximum of 10 bytes.

The following example decodes multiple varints from a buffer:

while (!data.empty()) {
  int64_t value;
  size_t bytes = Decode(data, &value);
 
  if (bytes == 0u) {
    return Status::DataLoss();
  }
  results.push_back(value);
  data = data.subspan(bytes)
}

Encode()

template<typename T >

size_t pw::varint::Encode	(	T	integer,
		const span< std::byte > &	output
	)

Encodes the provided integer using a variable-length encoding and returns the number of bytes written.

The encoding is the same as used in protocol buffers. Signed integers are ZigZag encoded to remove leading 1s from small negative numbers, then the resulting number is encoded as Little Endian Base 128 (LEB128). Unsigned integers are encoded directly as LEB128.

Returns the number of bytes written or 0 if the result didn't fit in the encoding buffer.

EncodedSize()

template<typename T , typename = std::enable_if_t<std::is_integral<T>::value || std::is_convertible<T, uint64_t>::value>>

constexpr size_t pw::varint::EncodedSize ( T  integer )

constexpr

Computes the size of an integer when encoded as a varint.

Parameters

integer

The integer whose encoded size is to be computed. integer can be signed or unsigned.

Returns

The size of integer when encoded as a varint.

EncodeLittleEndianBase128()

size_t pw::varint::EncodeLittleEndianBase128 ( uint64_t  integer, const span< std::byte > &  output  )

inline

Encodes a uint64_t with Little-Endian Base 128 (LEB128) encoding.

Returns

the number of bytes written; 0 if the buffer is too small

MaxValueInBytes()

constexpr uint64_t pw::varint::MaxValueInBytes ( size_t  bytes )

constexpr

Returns the maximum (max) integer value that can be encoded as a varint into the specified number of bytes.

The following table lists the max value for each byte size:

Bytes	Max value
1	127
2	16,383
3	2,097,151
4	268,435,455
5	34,359,738,367
6	4,398,046,511,103
7	562,949,953,421,311
8	72,057,594,037,927,935
9	9,223,372,036,854,775,807
10	(uint64 max value)

Parameters

bytes

The size of the varint, in bytes. 5 bytes are needed for the max uint32 value. 10 bytes are needed for the max uint64 value.

Returns

The max integer value for a varint of size bytes.

Read()

StatusWithSize pw::varint::Read	(	stream::Reader &	reader,
		int64_t *	output,
		size_t	max_size = std::numeric_limits< size_t >::max()
	)

Decodes a variable-length integer (varint) from the current position of a pw::stream. Reads a maximum of 10 bytes or max_size, whichever is smaller.

This method always returns the number of bytes read, even on error.

Parameters

reader	The pw::stream to read from.
output	The integer to read into. If reading into a signed integer, the value is ZigZag-decoded.
max_size	The maximum number of bytes to read. The upper limit is 10 bytes.

Returns

embed:rst:leading-asterisk
 
* 
*  .. pw-status-codes::
* 
*     OK: The decoded value is placed in ``output``.
* 
*     OUT_OF_RANGE: No input is available, e.g. the stream is closed.
* 
*     DATA_LOSS: The decoded value is too large for ``output`` or is
*     incomplete, e.g. the input was exhausted after a partial varint was
*     read.
* 
*  

ZigZagDecode()

template<typename T >

constexpr std::make_signed_t< T > pw::varint::ZigZagDecode ( T  n )

constexpr

ZigZag decodes a signed integer.

The calculation is done modulo std::numeric_limits<T>::max()+1, so the unsigned integer overflows are intentional.

ZigZagEncode()

template<typename T >

constexpr std::make_unsigned_t< T > pw::varint::ZigZagEncode ( T  n )

constexpr

ZigZag encodes a signed integer. This maps small negative numbers to small, unsigned positive numbers, which improves their density for LEB128 encoding.

ZigZag encoding works by moving the sign bit from the most-significant bit to the least-significant bit. For the signed k-bit integer n, the formula is:

(n << 1) ^ (n >> (k - 1))

See the following for a description of ZigZag encoding: https://protobuf.dev/programming-guides/encoding/#signed-ints