Packet protocol#

pw_rpc: Efficient, low-code-size RPC system for embedded devices

Pigweed RPC servers and clients communicate using pw_rpc packets. These packets are used to send requests and responses, control streams, cancel ongoing RPCs, and report errors.

Packet format#

Pigweed RPC packets consist of a type and a set of fields. The packets are encoded as protocol buffers. The full packet format is described in pw_rpc/pw_rpc/internal/packet.proto.

syntax = "proto3";

package pw.rpc.internal;

option java_package = "dev.pigweed.pw_rpc.internal";

enum PacketType {
  // To simplify identifying the origin of a packet, client-to-server packets
  // use even numbers and server-to-client packets use odd numbers.

  // Client-to-server packets

  // The client invokes an RPC. Always the first packet.
  REQUEST = 0;

  // A message in a client stream. Always sent after a REQUEST and before a
  // CLIENT_REQUEST_COMPLETION.
  CLIENT_STREAM = 2;

  // The client received a packet for an RPC it did not request.
  CLIENT_ERROR = 4;

  // Client has requested for call completion. In client streaming and
  // bi-directional streaming RPCs, this also indicates that the client is done
  // with sending requests.
  CLIENT_REQUEST_COMPLETION = 8;

  // Server-to-client packets

  // The RPC has finished.
  RESPONSE = 1;

  // The server was unable to process a request.
  SERVER_ERROR = 5;

  // A message in a server stream.
  SERVER_STREAM = 7;

  // Reserve field numbers for deprecated PacketTypes.
  reserved 3;  // SERVER_STREAM_END (equivalent to RESPONSE now)
  reserved 6;  // CANCEL (replaced by CLIENT_ERROR with status CANCELLED)
}

message RpcPacket {
  // The type of packet. Determines which other fields are used.
  PacketType type = 1;

  // Channel through which the packet is sent.
  uint32 channel_id = 2;

  // Hash of the fully-qualified name of the service with which this packet is
  // associated. For RPC packets, this is the service that processes the packet.
  fixed32 service_id = 3;

  // Hash of the name of the method which should process this packet.
  fixed32 method_id = 4;

  // The packet's payload, which is an encoded protobuf.
  bytes payload = 5;

  // Status code for the RPC response or error.
  uint32 status = 6;

  // Unique identifier for the call that initiated this RPC. Optionally set by
  // the client in the initial request and sent in all subsequent client
  // packets; echoed by the server.
  uint32 call_id = 7;
}

The packet type and RPC type determine which fields are present in a Pigweed RPC packet. Each packet type is only sent by either the client or the server. These tables describe the meaning of and fields included with each packet type.

Client-to-server packets#

packet type

description

REQUEST

Invoke an RPC

- channel_id
- service_id
- method_id
- payload
  (unary & server streaming only)
- call_id (optional)

CLIENT_STREAM

Message in a client stream

- channel_id
- service_id
- method_id
- payload
- call_id (if set in REQUEST)

CLIENT_REQUEST_COMPLETION

Client requested stream completion

- channel_id
- service_id
- method_id
- call_id (if set in REQUEST)

CLIENT_ERROR

Abort an ongoing RPC

- channel_id
- service_id
- method_id
- status
- call_id (if set in REQUEST)

Client errors

The client sends CLIENT_ERROR packets to a server when it receives a packet it did not request. If possible, the server should abort it.

The status code indicates the type of error. The status code is logged, but all status codes result in the same action by the server: aborting the RPC.

  • CANCELLED – The client requested that the RPC be cancelled.

  • ABORTED – The RPC was aborted due its channel being closed.

  • NOT_FOUND – Received a packet for a service method the client does not recognize.

  • FAILED_PRECONDITION – Received a packet for a service method that the client did not invoke.

  • DATA_LOSS – Received a corrupt packet for a pending service method.

  • INVALID_ARGUMENT – The server sent a packet type to an RPC that does not support it (a SERVER_STREAM was sent to an RPC with no server stream).

  • UNAVAILABLE – Received a packet for an unknown channel.

Server-to-client packets#

packet type

description

RESPONSE

The RPC is complete

- channel_id
- service_id
- method_id
- status
- payload
  (unary & client streaming only)
- call_id (if set in REQUEST)

SERVER_STREAM

Message in a server stream

- channel_id
- service_id
- method_id
- payload
- call_id (if set in REQUEST)

SERVER_ERROR

Received unexpected packet

- channel_id
- service_id (if relevant)
- method_id (if relevant)
- status
- call_id (if set in REQUEST)

All server packets contain the same call_id that was set in the initial request made by the client, if any.

Server errors

The server sends SERVER_ERROR packets when it receives a packet it cannot process. The client should abort any RPC for which it receives an error. The status field indicates the type of error.

  • NOT_FOUND – The requested service or method does not exist.

  • FAILED_PRECONDITION – A client stream or cancel packet was sent for an RPC that is not pending.

  • INVALID_ARGUMENT – The client sent a packet type to an RPC that does not support it (a CLIENT_STREAM was sent to an RPC with no client stream).

  • RESOURCE_EXHAUSTED – The request came on a new channel, but a channel could not be allocated for it.

  • ABORTED – The RPC was aborted due its channel being closed.

  • INTERNAL – The server was unable to respond to an RPC due to an unrecoverable internal error.

  • UNAVAILABLE – Received a packet for an unknown channel.

Invoking a service method#

Calling an RPC requires a specific sequence of packets. This section describes the protocol for calling service methods of each type: unary, server streaming, client streaming, and bidirectional streaming.

The basic flow for all RPC invocations is as follows:

  • Client sends a REQUEST packet. Includes a payload for unary & server streaming RPCs.

  • For client and bidirectional streaming RPCs, the client may send any number of CLIENT_STREAM packets with payloads.

  • For server and bidirectional streaming RPCs, the server may send any number of SERVER_STREAM packets.

  • The server sends a RESPONSE packet. Includes a payload for unary & client streaming RPCs. The RPC is complete.

The client may cancel an ongoing RPC at any time by sending a CLIENT_ERROR packet with status CANCELLED. The server may finish an ongoing RPC at any time by sending the RESPONSE packet.

Unary RPC#

In a unary RPC, the client sends a single request and the server sends a single response.

sequenceDiagram participant C as Client participant S as Server C->>+S: request Note left of C: PacketType.REQUEST<br>channel ID<br>service ID<br>method ID<br>payload S->>-C: response Note right of S: PacketType.RESPONSE<br>channel ID<br>service ID<br>method ID<br>payload<br>status

The client may attempt to cancel a unary RPC by sending a CLIENT_ERROR packet with status CANCELLED. The server sends no response to a cancelled RPC. If the server processes the unary RPC synchronously (the handling thread sends the response), it may not be possible to cancel the RPC.

sequenceDiagram participant C as Client participant S as Server C->>+S: request Note left of C: PacketType.REQUEST<br>channel ID<br>service ID<br>method ID<br>payload C->>S: cancel Note left of C: PacketType.CLIENT_ERROR<br>channel ID<br>service ID<br>method ID<br>status=CANCELLED

Server streaming RPC#

In a server streaming RPC, the client sends a single request and the server sends any number of SERVER_STREAM packets followed by a RESPONSE packet.

sequenceDiagram participant C as Client participant S as Server C->>+S: request Note left of C: PacketType.REQUEST<br>channel ID<br>service ID<br>method ID<br>payload S-->>C: messages (zero or more) Note right of S: PacketType.SERVER_STREAM<br>channel ID<br>service ID<br>method ID<br>payload S->>-C: done Note right of S: PacketType.RESPONSE<br>channel ID<br>service ID<br>method ID<br>status

The client may terminate a server streaming RPC by sending a CLIENT_STREAM packet with status CANCELLED. The server sends no response.

sequenceDiagram participant C as Client participant S as Server C->>S: request Note left of C: PacketType.REQUEST<br>channel ID<br>service ID<br>method ID<br>payload S-->>C: messages (zero or more) Note right of S: PacketType.SERVER_STREAM<br>channel ID<br>service ID<br>method ID<br>payload C->>S: cancel Note left of C: PacketType.CLIENT_ERROR<br>channel ID<br>service ID<br>method ID<br>status=CANCELLED

Client streaming RPC#

In a client streaming RPC, the client starts the RPC by sending a REQUEST packet with no payload. It then sends any number of messages in CLIENT_STREAM packets, followed by a CLIENT_REQUEST_COMPLETION. The server sends a single RESPONSE to finish the RPC.

sequenceDiagram participant C as Client participant S as Server C->>S: start Note left of C: PacketType.REQUEST<br>channel ID<br>service ID<br>method ID<br>payload C-->>S: messages (zero or more) Note left of C: PacketType.CLIENT_STREAM<br>channel ID<br>service ID<br>method ID<br>payload C->>S: done Note left of C: PacketType.CLIENT_REQUEST_COMPLETION<br>channel ID<br>service ID<br>method ID S->>C: response Note right of S: PacketType.RESPONSE<br>channel ID<br>service ID<br>method ID<br>payload<br>status

The server may finish the RPC at any time by sending its RESPONSE packet, even if it has not yet received the CLIENT_REQUEST_COMPLETION packet. The client may terminate the RPC at any time by sending a CLIENT_ERROR packet with status CANCELLED.

sequenceDiagram participant C as Client participant S as Server C->>S: start Note left of C: PacketType.REQUEST<br>channel ID<br>service ID<br>method ID C-->>S: messages (zero or more) Note left of C: PacketType.CLIENT_STREAM<br>channel ID<br>service ID<br>method ID<br>payload C->>S: cancel Note right of S: PacketType.CLIENT_ERROR<br>channel ID<br>service ID<br>method ID<br>status=CANCELLED

Bidirectional streaming RPC#

In a bidirectional streaming RPC, the client sends any number of requests and the server sends any number of responses. The client invokes the RPC by sending a REQUEST with no payload. It sends a CLIENT_REQUEST_COMPLETION packet when it has finished sending requests. The server sends a RESPONSE packet to finish the RPC.

sequenceDiagram participant C as Client participant S as Server C->>S: start Note left of C: PacketType.REQUEST<br>channel ID<br>service ID<br>method ID<br>payload C-->>S: messages (zero or more) Note left of C: PacketType.CLIENT_STREAM<br>channel ID<br>service ID<br>method ID<br>payload C-->S: (messages in any order) S-->>C: messages (zero or more) Note right of S: PacketType.SERVER_STREAM<br>channel ID<br>service ID<br>method ID<br>payload C->>S: done Note left of C: PacketType.CLIENT_REQUEST_COMPLETION<br>channel ID<br>service ID<br>method ID S->>C: done Note right of S: PacketType.RESPONSE<br>channel ID<br>service ID<br>method ID<br>status

The server may finish the RPC at any time by sending the RESPONSE packet, even if it has not received the CLIENT_REQUEST_COMPLETION packet. The client may terminate the RPC at any time by sending a CLIENT_ERROR packet with status CANCELLED.

sequenceDiagram participant C as Client participant S as Server C->>S: start Note left of C: PacketType.REQUEST<br>channel ID<br>service ID<br>method ID<br>payload C-->>S: messages (zero or more) Note left of C: PacketType.CLIENT_STREAM<br>channel ID<br>service ID<br>method ID<br>payload C->>S: done Note left of C: PacketType.CLIENT_REQUEST_COMPLETION<br>channel ID<br>service ID<br>method ID S->>C: response Note right of S: PacketType.RESPONSE<br>channel ID<br>service ID<br>method ID<br>payload<br>status