API reference#
pw_hdlc: Simple, robust, and efficient serial communication
The pw_hdlc
API has 3 conceptual parts:
Encoder: Encode data as HDLC unnumbered information frames.
Decoder: Decode HDLC frames from a stream of data.
RPC: Use RPC over HDLC.
Encoder#
Single-Function Encoding#
Pigweed offers a single function which will encode an HDLC frame in each of C++, Python, and TypeScript:
-
Status pw::hdlc::WriteUIFrame(uint64_t address, ConstByteSpan payload, stream::Writer &writer)#
Writes an HDLC unnumbered information frame (UI frame) to the provided
pw::stream
writer.This function is a convenience alias for the more general
Encoder
type and set of functions.- Parameters:
address – The frame address.
payload – The frame data to encode.
writer – The
pw::stream
to write the frame to. The frame contains the following bytes. See Design for more information.HDLC flag byte (
0x7e
)Address (variable length, up to 10 bytes)
UI-frame control (metadata) byte
Payload (0 or more bytes)
Frame check sequence (CRC-32, 4 bytes)
HDLC flag byte (
0x7e
)
- Returns:
A
pw::Status
instance describing the result of the operation:OK
- The write finished successfully.RESOURCE_EXHAUSTED
- The write failed because the size of the frame would be larger than the writer’s conservative limit.INVALID_ARGUMENT
- The start of the write failed. Check for problems in youraddress
argument’s value.
Example:
// Writes a span of data to a pw::stream::Writer and returns the status. This
// implementation uses the pw_checksum module to compute the CRC-32 frame check
// sequence.
#include "pw_hdlc/encoder.h"
#include "pw_hdlc/sys_io_stream.h"
int main() {
pw::stream::SysIoWriter serial_writer;
Status status = WriteUIFrame(123 /* address */, data, serial_writer);
if (!status.ok()) {
PW_LOG_INFO("Writing frame failed! %s", status.str());
}
}
The encode module supports encoding HDLC frames.
- pw_hdlc.encode.ui_frame(address: int, data: bytes) bytes
Encodes an HDLC UI-frame with a CRC-32 frame check sequence.
Example:
# Read bytes from serial and encode HDLC frames
import serial
from pw_hdlc import encode
ser = serial.Serial()
address = 123
ser.write(encode.ui_frame(address, b'your data here!'))
Encoder
provides a way to build complete, escaped HDLC unnumbered
information frames.
- Encoder.uiFrame(address, data)
- Arguments:
address (
number()
) – frame address.data (
Uint8Array()
) – frame data.
- Returns:
Uint8Array
containing a complete HDLC frame.
Piecemeal Encoding#
Additionally, the C++ API provides an API for piecemeal encoding of an HDLC frame. This allows frames to be encoded gradually without ever holding an entire frame in memory at once.
-
class Encoder#
Encodes and writes HDLC frames.
Decoder#
-
class Decoder#
Subclassed by pw::hdlc::DecoderBuffer< kSizeBytes >
Public Functions
-
Result<Frame> Process(std::byte new_byte)#
Parses a single byte of an HDLC stream.
- Returns:
A
pw::Result
with the complete frame if the byte completes a frame. The status can be one of the following:OK
- A frame was successfully decoded. TheResult
contains theFrame
, which is invalidated by the nextProcess()
call.UNAVAILABLE
- No frame is available.RESOURCE_EXHAUSTED
- A frame completed, but it was too large to fit in the decoder’s buffer.DATA_LOSS
- A frame completed, but it was invalid. The frame was incomplete or the frame check sequence verification failed.
-
Result<Frame> Process(std::byte new_byte)#
Example:
// Read individual bytes from pw::sys_io and decode HDLC frames.
#include "pw_hdlc/decoder.h"
#include "pw_sys_io/sys_io.h"
int main() {
std::byte data;
while (true) {
if (!pw::sys_io::ReadByte(&data).ok()) {
// Log serial reading error
}
Result<Frame> decoded_frame = decoder.Process(data);
if (decoded_frame.ok()) {
// Handle the decoded frame
}
}
}
- class pw_hdlc.decode.FrameDecoder
Decodes one or more HDLC frames from a stream of data.
- __init__() None
- process(data: bytes) Iterable[Frame]
Decodes and yields HDLC frames, including corrupt frames.
The
ok()
method onFrame
indicates whether it is valid or represents a frame parsing error.- Yields:
Frames, which may be valid (
frame.ok()
) or corrupt (!frame.ok()
)
- process_byte(byte: int) Frame | None
Processes a single byte and returns a frame if one was completed.
- process_valid_frames(data: bytes) Iterable[Frame]
Decodes and yields valid HDLC frames, logging any errors.
Example:
# Decode data read from serial
import serial
from pw_hdlc import decode
ser = serial.Serial()
decoder = decode.FrameDecoder()
while True:
for frame in decoder.process_valid_frames(ser.read()):
# Handle the decoded frame
It is possible to decode HDLC frames from a stream using different protocols or unstructured data. This is not recommended, but may be necessary when introducing HDLC to an existing system.
The FrameAndNonFrameDecoder
Python class supports working with raw data and
HDLC frames in the same stream.
- class pw_hdlc.decode.FrameAndNonFrameDecoder(
- non_frame_data_handler: Callable[[bytes], Any],
- *,
- mtu: int | None = None,
- timeout_s: float | None = None,
- handle_shared_flags: bool = True,
Processes both HDLC frames and non-frame data in a stream.
- __init__(
- non_frame_data_handler: Callable[[bytes], Any],
- *,
- mtu: int | None = None,
- timeout_s: float | None = None,
- handle_shared_flags: bool = True,
Yields valid HDLC frames and passes non-frame data to callback.
- Parameters:
mtu – Maximum bytes to receive before flushing raw data. If a valid HDLC frame contains more than MTU bytes, the valid frame will be emitted, but part of the frame will be included in the raw data.
timeout_s – How long to wait before automatically flushing raw data. If a timeout occurs partway through a valid frame, the frame will be emitted, but part of the frame will be included in the raw data.
handle_shared_flags – Whether to permit HDLC frames to share a single flag byte between frames. If
False
, partial HDLC frames may be emitted as raw data when HDLC frames share a flag byte, but raw data won’t have to wait for a timeout or full MTU to be flushed.
- flush_non_frame_data() None
Flushes any data in the buffer as non-frame data.
If a valid HDLC frame was flushed partway, the data for the first part of the frame will be included both in the raw data and in the frame.
- process(data: bytes) Iterable[Frame]
Processes a stream of mixed HDLC and unstructured data.
Yields OK frames and calls
non_frame_data_handler()
with non-HDLC data.
Decoder
unescapes received bytes and adds them to a buffer. Complete,
valid HDLC frames are yielded as they are received.
- Decoder.process(data)
- Arguments:
data (
Uint8Array()
) – bytes to be decoded.
- Yields:
HDLC frames, including corrupt frames. The
Frame.ok()
method whether the frame is valid.
- processValidFrames(data)
- Arguments:
data (
Uint8Array()
) – bytes to be decoded.
- Yields:
Valid HDLC frames, logging any errors.
RPC#
RpcChannelOutput
implements the pw::rpc::ChannelOutput
interface
of pw_rpc
, simplifying the process of creating an RPC channel over HDLC.
A pw::stream::Writer
must be provided as the underlying transport
implementation.
If your HDLC routing path has a Maximum Transmission Unit (MTU) limitation,
use the FixedMtuChannelOutput
to verify that the currently configured
max RPC payload size (dictated by the static encode buffer of pw_rpc
)
will always fit safely within the limits of the fixed HDLC MTU after
HDLC encoding.
The pw_hdlc
Python package includes utilities to HDLC-encode and
decode RPC packets, with examples of RPC client implementations in Python.
It also provides abstractions for interfaces used to receive RPC Packets.
The pw_hdlc.rpc.CancellableReader
and pw_hdlc.rpc.RpcClient
classes and derived classes are context-managed to cleanly cancel the
read process and stop the reader thread. The pw_hdlc.rpc.SocketReader
and pw_hdlc.rpc.SerialReader
also close the provided interface on
context exit. It is recommended to use these in a context statement. For
example:
import serial
from pw_hdlc import rpc
if __name__ == '__main__':
serial_device = serial.Serial('/dev/ttyACM0')
with rpc.SerialReader(serial_device) as reader:
with rpc.HdlcRpcClient(
reader,
[],
rpc.default_channels(serial_device.write)) as rpc_client:
# Do something with rpc_client.
# The serial_device object is closed, and reader thread stopped.
return 0
- class pw_hdlc.rpc.channel_output(writer: Callable[[bytes], Any], address: int = 82, delay_s: float = 0)
Returns a function that can be used as a channel output for
pw_rpc
.
- class pw_hdlc.rpc.CancellableReader(base_obj: Any, *read_args, **read_kwargs)
Wraps communication interfaces used for reading incoming data with the guarantee that the read request can be cancelled. Derived classes must implement the
cancel_read()
method.Cancelling a read invalidates ongoing and future reads. The
cancel_read()
method can only be called once.- __init__(base_obj: Any, *read_args, **read_kwargs)
- Parameters:
base_obj – Object that offers a
read()
method with optional args and kwargs.read_args – Arguments for
base_obj.read()
function.read_kwargs – Keyword arguments for
base_obj.read()
function.
- abstract cancel_read() None
Cancels a blocking read request and all future reads.
Can only be called once.
- read() bytes
Reads bytes that contain parts of or full RPC packets.
- class pw_hdlc.rpc.SelectableReader(base_obj: Any, *read_args, **read_kwargs)
Wraps interfaces that work with
select()
to signal when data is received.These interfaces must provide a
fileno()
method. WINDOWS ONLY: Only sockets that originate from WinSock can be wrapped. File objects are not acceptable.- __init__(base_obj: Any, *read_args, **read_kwargs)
- Parameters:
base_obj – Object that offers a
read()
method with optional args and kwargs.read_args – Arguments for
base_obj.read()
function.read_kwargs – Keyword arguments for
base_obj.read()
function.
- cancel_read() None
Cancels a blocking read request and all future reads.
Can only be called once.
- read() bytes
Reads bytes that contain parts of or full RPC packets.
- class pw_hdlc.rpc.SocketReader(base_obj: socket, *read_args, **read_kwargs)
Wraps a socket
recv()
function.- __init__(base_obj: socket, *read_args, **read_kwargs)
- Parameters:
base_obj – Object that offers a
read()
method with optional args and kwargs.read_args – Arguments for
base_obj.read()
function.read_kwargs – Keyword arguments for
base_obj.read()
function.
- read() bytes
Reads bytes that contain parts of or full RPC packets.
- class pw_hdlc.rpc.SerialReader(base_obj: Serial, *read_args, **read_kwargs)
Wraps a
serial.Serial
object.- __init__(base_obj: Serial, *read_args, **read_kwargs)
- Parameters:
base_obj – Object that offers a
read()
method with optional args and kwargs.read_args – Arguments for
base_obj.read()
function.read_kwargs – Keyword arguments for
base_obj.read()
function.
- cancel_read() None
Cancels a blocking read request and all future reads.
Can only be called once.
- class pw_hdlc.rpc.DataReaderAndExecutor(
- reader: CancellableReader,
- on_read_error: Callable[[Exception], None],
- data_processor: Callable[[bytes], Iterable[FrameTypeT]],
- frame_handler: Callable[[FrameTypeT], None],
- handler_threads: int | None = 1,
Reads incoming bytes, data processor that delegates frame handling.
Executing callbacks in a
ThreadPoolExecutor
decouples reading the input stream from handling the data. That way, if a handler function takes a long time or crashes, this reading thread is not interrupted.- __init__(
- reader: CancellableReader,
- on_read_error: Callable[[Exception], None],
- data_processor: Callable[[bytes], Iterable[FrameTypeT]],
- frame_handler: Callable[[FrameTypeT], None],
- handler_threads: int | None = 1,
Creates the data reader and frame delegator.
- Parameters:
reader – Reads incoming bytes from the given transport, blocks until data is available or an exception is raised. Otherwise the reader will exit.
on_read_error – Called when there is an error reading incoming bytes.
data_processor – Processes read bytes and returns a frame-like object that the frame_handler can process.
frame_handler – Handles a received frame.
handler_threads – The number of threads in the executor pool.
- start() None
Starts the reading process.
- stop() None
Stops the reading process.
This requests that the reading process stop and waits for the background thread to exit.
- class pw_hdlc.rpc.default_channels(write: Callable[[bytes], Any])
- class pw_hdlc.rpc.RpcClient(
- reader_and_executor: DataReaderAndExecutor,
- paths_or_modules: Iterable[str | Path | module] | Library,
- channels: Iterable[Channel],
- client_impl: ClientImpl | None = None,
An RPC client with configurable incoming data processing.
- __init__(
- reader_and_executor: DataReaderAndExecutor,
- paths_or_modules: Iterable[str | Path | module] | Library,
- channels: Iterable[Channel],
- client_impl: ClientImpl | None = None,
Creates an RPC client.
- Parameters:
reader_and_executor –
DataReaderAndExecutor
instance.paths_or_modules – paths to .proto files or proto modules.
channels – RPC channels to use for output.
client_impl – The RPC client implementation. Defaults to the callback client implementation if not provided.
- rpcs(channel_id: int | None = None) Any
Returns object for accessing services on the specified channel.
This skips some intermediate layers to make it simpler to invoke RPCs from an
HdlcRpcClient
. If only one channel is in use, the channel ID is not necessary.
- class pw_hdlc.rpc.HdlcRpcClient(reader: ~pw_hdlc.rpc.CancellableReader, paths_or_modules: ~typing.Iterable[str | ~pathlib.Path | module] | ~pw_protobuf_compiler.python_protos.Library, channels: ~typing.Iterable[~pw_rpc.descriptors.Channel], output: ~typing.Callable[[bytes], ~typing.Any] = <function write_to_file>, client_impl: ~pw_rpc.client.ClientImpl | None = None, *, _incoming_packet_filter_for_testing: ~pw_rpc.descriptors.ChannelManipulator | None = None, rpc_frames_address: int = 82, log_frames_address: int = 1, extra_frame_handlers: dict[int, typing.Callable[[pw_hdlc.decode.Frame], typing.Any]] | None = None)
An RPC client configured to run over HDLC.
Expects HDLC frames to have addresses that dictate how to parse the HDLC payloads.
- __init__(reader: ~pw_hdlc.rpc.CancellableReader, paths_or_modules: ~typing.Iterable[str | ~pathlib.Path | module] | ~pw_protobuf_compiler.python_protos.Library, channels: ~typing.Iterable[~pw_rpc.descriptors.Channel], output: ~typing.Callable[[bytes], ~typing.Any] = <function write_to_file>, client_impl: ~pw_rpc.client.ClientImpl | None = None, *, _incoming_packet_filter_for_testing: ~pw_rpc.descriptors.ChannelManipulator | None = None, rpc_frames_address: int = 82, log_frames_address: int = 1, extra_frame_handlers: dict[int, typing.Callable[[pw_hdlc.decode.Frame], typing.Any]] | None = None)
Creates an RPC client configured to communicate using HDLC.
- Parameters:
reader – Readable object used to receive RPC packets.
paths_or_modules – paths to .proto files or proto modules.
channels – RPC channels to use for output.
output – where to write
stdout
output from the device.client_impl – The RPC Client implementation. Defaults to the callback client implementation if not provided.
rpc_frames_address – the address used in the HDLC frames for RPC packets. This can be the channel ID, or any custom address.
log_frames_address – the address used in the HDLC frames for
stdout
output from the device.extra_fram_handlers – Optional mapping of HDLC frame addresses to their callbacks.
- class pw_hdlc.rpc.NoEncodingSingleChannelRpcClient(
- reader: CancellableReader,
- paths_or_modules: Iterable[str | Path | module] | Library,
- channel: Channel,
- client_impl: ClientImpl | None = None,
An RPC client without any frame encoding with a single channel output.
The caveat is that the provided read function must read entire frames.
- __init__(
- reader: CancellableReader,
- paths_or_modules: Iterable[str | Path | module] | Library,
- channel: Channel,
- client_impl: ClientImpl | None = None,
Creates an RPC client over a single channel with no frame encoding.
- Parameters:
reader – Readable object used to receive RPC packets.
paths_or_modules – paths to .proto files or proto modules.
channel – RPC channel to use for output.
client_impl – The RPC Client implementation. Defaults to the callback client implementation if not provided.
- class pw_hdlc.rpc.SocketSubprocess(command: Sequence, port: int)
Executes a subprocess and connects to it with a socket.
- __init__(command: Sequence, port: int) None
- class pw_hdlc.rpc.HdlcRpcLocalServerAndClient(
- server_command: Sequence,
- port: int,
- protos: Iterable[str | Path | module] | Library,
- *,
- incoming_processor: ChannelManipulator | None = None,
- outgoing_processor: ChannelManipulator | None = None,
Runs an RPC server in a subprocess and connects to it over a socket.
This can be used to run a local RPC server in an integration test.
- __init__(
- server_command: Sequence,
- port: int,
- protos: Iterable[str | Path | module] | Library,
- *,
- incoming_processor: ChannelManipulator | None = None,
- outgoing_processor: ChannelManipulator | None = None,
Creates a new
HdlcRpcLocalServerAndClient
.
The TypeScript library doesn’t have an RPC interface.