pw_web#

Pigweed provides an NPM package with modules to build web apps for Pigweed devices.

Getting Started#

Easiest way to get started is to follow the Sense tutorial and flash a Raspberry Pico board.

Once you have a device running Pigweed, you can connect to it using just your web browser.

Installation#

If you have a bundler set up, you can install pigweedjs in your web application by:

$ npm install --save pigweedjs

After installing, you can import modules from pigweedjs in this way:

import { pw_rpc, pw_tokenizer, Device, WebSerial } from 'pigweedjs';

Import Directly in HTML#

If you don’t want to set up a bundler, you can also load Pigweed directly in your HTML page by:

<script src="https://unpkg.com/pigweedjs/dist/index.umd.js"></script>
<script>
  const { pw_rpc, pw_hdlc, Device, WebSerial } from Pigweed;
</script>

Modules#

Device#

Device class is a helper API to connect to a device over serial and call RPCs easily.

To initialize device, it needs a ProtoCollection instance. pigweedjs includes a default one which you can use to get started, you can also generate one from your own .proto files using pw_proto_compiler.

Device goes through all RPC methods in the provided ProtoCollection. For each RPC, it reads all the fields in Request proto and generates a JavaScript function to call that RPC and also a helper method to create a request. It then makes this function available under rpcs.* namespaced by its package name.

Device has following public API:

constructor(ProtoCollection, WebSerialTransport <optional>, channel <optional>, rpcAddress <optional>)
connect() - Shows browser’s WebSerial connection dialog and let’s user make device selection
rpcs.* - Device API enumerates all RPC services and methods present in the provided proto collection and makes them available as callable functions under rpcs. Example: If provided proto collection includes Pigweed’s Echo service ie. pw.rpc.EchoService.Echo, it can be triggered by calling device.rpcs.pw.rpc.EchoService.Echo.call(request). The functions return a Promise that resolves an array with status and response.

Using Device API with Sense#

Sense project uses pw_log_rpc; an RPC-based logging solution. Sense also uses pw_tokenizer to tokenize strings and save device space. Below is an example that streams logs using the Device API.

<h1>Hello Pigweed</h1>
<button onclick="connect()">Connect</button>
<br /><br />
<code></code>
<script src="https://unpkg.com/pigweedjs/dist/index.umd.js"></script>
<script src="https://unpkg.com/pigweedjs/dist/protos/collection.umd.js"></script>
<script>
  const { Device, pw_tokenizer } = Pigweed;
  const { ProtoCollection } = PigweedProtoCollection;
  const tokenDBCsv = `...` // Load token database here

  const device = new Device(new ProtoCollection());
  const detokenizer = new pw_tokenizer.Detokenizer(tokenDBCsv);

  async function connect(){
    await device.connect();
    const req = device.rpcs.pw.log.Logs.Listen.createRequest()
    const logs = device.rpcs.pw.log.Logs.Listen.call(req);
    for await (const msg of logs){
        msg.getEntriesList().forEach((entry) => {
          const frame = entry.getMessage();
          const detokenized = detokenizer.detokenizeUint8Array(frame);
          document.querySelector('code').innerHTML += detokenized + "<br/>";
        });
    }
    console.log("Log stream ended with status", logs.call.status);
  }
</script>

The above example requires a token database in CSV format. You can generate one from the Sense’s .elf file by running:

$ pw_tokenizer/py/pw_tokenizer/database.py create \
--database db.csv bazel-bin/apps/blinky/rp2040_blinky.elf

You can then load this CSV in JavaScript using fetch() or by just copying the contents into the tokenDBCsv variable in the above example.

WebSerialTransport#

To help with connecting to WebSerial and listening for serial data, a helper class is also included under WebSerial.WebSerialTransport. Here is an example usage:

import { WebSerial, pw_hdlc } from 'pigweedjs';

const transport = new WebSerial.WebSerialTransport();
const decoder = new pw_hdlc.Decoder();

// Present device selection prompt to user
await transport.connect();

// Or connect to an existing `SerialPort`
// await transport.connectPort(port);

// Listen and decode HDLC frames
transport.chunks.subscribe((item) => {
  const decoded = decoder.process(item);
  for (const frame of decoded) {
    if (frame.address === 1) {
      const decodedLine = new TextDecoder().decode(frame.data);
      console.log(decodedLine);
    }
  }
});

// Later, close all streams and close the port.
transport.disconnect();

Individual Modules#

Following Pigweed modules are included in the NPM package:

Log Viewer Component#

The NPM package also includes a log viewer component that can be embedded in any webapp. The component works with Pigweed’s RPC stack out-of-the-box but also supports defining your own log source. See Log viewer for component interaction details.

The component is composed of the component itself and a log source. Here is a simple example app that uses a mock log source:

<div id="log-viewer-container"></div>
<script src="https://unpkg.com/pigweedjs/dist/logging.umd.js"></script>
<script>

  const { createLogViewer, MockLogSource } = PigweedLogging;
  const logSource = new MockLogSource();
  const containerEl = document.querySelector(
    '#log-viewer-container'
  );

  let unsubscribe = createLogViewer(logSource, containerEl);
  logSource.start(); // Start producing mock logs

</script>

The code above will render a working log viewer that just streams mock log entries.

It also comes with an RPC log source with support for detokenization. Here is an example app using that:

<div id="log-viewer-container"></div>
<script src="https://unpkg.com/pigweedjs/dist/index.umd.js"></script>
<script src="https://unpkg.com/pigweedjs/dist/protos/collection.umd.js"></script>
<script src="https://unpkg.com/pigweedjs/dist/logging.umd.js"></script>
<script>

  const { Device, pw_tokenizer } = Pigweed;
  const { ProtoCollection } = PigweedProtoCollection;
  const { createLogViewer, PigweedRPCLogSource } = PigweedLogging;

  const device = new Device(new ProtoCollection());
  const logSource = new PigweedRPCLogSource(device, "CSV TOKEN DB HERE");
  const containerEl = document.querySelector(
    '#log-viewer-container'
  );

  let unsubscribe = createLogViewer(logSource, containerEl);

</script>

Custom Log Source#

You can define a custom log source that works with the log viewer component by just extending the abstract LogSource class and emitting the logEntry events like this:

import { LogSource, LogEntry, Level } from 'pigweedjs/logging';

export class MockLogSource extends LogSource {
  constructor(){
    super();
    // Do any initializations here
    // ...
    // Then emit logs
    const log1: LogEntry = {

    }
    this.publishLogEntry({
      level: Level.INFO,
      timestamp: new Date(),
      fields: [
        { key: 'level', value: level }
        { key: 'timestamp', value: new Date().toISOString() },
        { key: 'source', value: "LEFT SHOE" },
        { key: 'message', value: "Running mode activated." }
      ]
    });
  }
}

After this, you just need to pass your custom log source object to createLogViewer(). See implementation of PigweedRPCLogSource for reference.

Column Order#

Column Order can be defined on initialization with the optional columnOrder parameter. Only fields that exist in the Log Source will render as columns in the Log Viewer.

createLogViewer(logSource, root, { columnOrder })

columnOrder accepts an string[] and defaults to [log_source, time, timestamp]

 createLogViewer(
  logSource,
  root,
  { columnOrder: ['log_source', 'time', 'timestamp'] }

)

Note, columns will always start with level and end with message, these fields do not need to be defined. Columns are ordered in the following format:

level
columnOrder
Fields that exist in Log Source but not listed will be added here.
message

Accessing and Modifying Log Views#

It can be challenging to access and manage log views directly through JavaScript or HTML due to the shadow DOM boundaries generated by custom elements. To facilitate this, the Log Viewer component has a public property, logViews, which returns an array containing all child log views. Here is an example that modifies the viewTitle and searchText properties of two log views:

const logViewer = containerEl.querySelector('log-viewer');
const views = logViewer?.logViews;

if (views) {
  views[0].viewTitle = 'Device A Logs';
  views[0].searchText = 'device:A';

  views[1].viewTitle = 'Device B Logs';
  views[1].searchText = 'device:B';
}

Alternatively, you can define a state object containing nodes with their respective properties and pass this state object to the Log Viewer during initialization. Here is how you can achieve that:

const childNodeA: ViewNode = new ViewNode({
  type: NodeType.View,
  viewTitle: 'Device A Logs',
  searchText: 'device:A'
});

const childNodeB: ViewNode = new ViewNode({
  type: NodeType.View,
  viewTitle: 'Device B Logs',
  searchText: 'device:B'
});

const rootNode: ViewNode = new ViewNode({
  type: NodeType.Split,
  orientation: Orientation.Vertical,
  children: [childNodeA, childNodeB]
});

const options = { state: { rootNode: rootNode } };
createLogViewer(logSources, containerEl, options);

Note that the relevant types and enums should be imported from log-viewer/src/shared/view-node.ts.

Color Scheme#

The log viewer web component provides the ability to set the color scheme manually, overriding any default or system preferences.

To set the color scheme, first obtain a reference to the log-viewer element in the DOM. A common way to do this is by using querySelector():

const logViewer = document.querySelector('log-viewer');

You can then set the color scheme dynamically by updating the component’s colorScheme property or by setting a value for the colorscheme HTML attribute.

logViewer.colorScheme = 'dark';

logViewer.setAttribute('colorscheme', 'dark');

The color scheme can be set to 'dark', 'light', or the default 'auto' which allows the component to adapt to the preferences in the operating system settings.

Material Icon Font (Subsetting)#

The Log Viewer uses a subset of the Material Symbols Rounded icon font fetched via the Google Fonts API. However, we also provide a subset of this font for offline usage at GitHub with codepoints listed in the codepoints file.

(It’s easiest to look up the codepoints at fonts.google.com e.g. see the sidebar shows the Codepoint for “home” is e88a).

The following icons with codepoints are curently used:

delete_sweep e16c
error e000
warning f083
cancel e5c9
bug_report e868
info e88e
view_column e8ec
brightness_alert f5cf
wrap_text e25b
more_vert e5d4
play_arrow e037
stop e047

To save load time and bandwidth, we provide a pre-made subset of the font with just the codepoints we need, which reduces the font size from 3.74MB to 12KB.

We use fonttools (fonttools/fonttools) to create the subset. To create your own subset, find the codepoints you want to add and:

Start a python virtualenv and install fonttools

virtualenv env
source env/bin/activate
pip install fonttools brotli

Download the the raw MaterialSybmolsRounded woff2 file

# line below for example, the url is not stable: e.g.
curl -L -o MaterialSymbolsRounded.woff2 \
  "https://github.com/google/material-design-icons/raw/master/variablefont/MaterialSymbolsRounded%5BFILL,GRAD,opsz,wght%5D.woff2"

Run fonttools, passing in the unicode codepoints of the necessary glyphs. (The points for letters a-z, numbers 0-9 and underscore character are necessary for creating ligatures)

Warning

Ensure there are no spaces in the list of codepoints.

fonttools subset MaterialSymbolsRounded.woff2 \
   --unicodes=5f-7a,30-39,e16c,e000,e002,e8b2,e5c9,e868,,e88e,e8ec,f083,f5cf,e25b,e5d4,e037,e047 \
   --no-layout-closure \
   --output-file=material_symbols_rounded_subset.woff2 \
   --flavor=woff2

Update material_symbols_rounded_subset.woff2 in log_viewer/src/assets with the new subset

Shoelace#

We currently use Split Panel from the Shoelace library to enable resizable split views within the log viewer.

To provide flexibility in different environments, we’ve introduced a property useShoelaceFeatures in the LogViewer component. This flag allows developers to enable or disable the import and usage of Shoelace components based on their needs.

By default, the useShoelaceFeatures flag is set to true, meaning Shoelace components will be used and resizable split views are made available. To disable Shoelace components, set this property to false as shown below:

const logViewer = document.querySelector('log-viewer');
logViewer.useShoelaceFeatures = false;

When useShoelaceFeatures is set to false, the <sl-split-panel> component from Shoelace will not be imported or used within the log viewer.

pw_web#

Getting Started#

Installation#

Import Directly in HTML#

Modules#

Device#

Using Device API with Sense#

WebSerialTransport#

Individual Modules#

Log Viewer Component#

Custom Log Source#

Column Order#

Accessing and Modifying Log Views#

Color Scheme#

Material Icon Font (Subsetting)#

Shoelace#

Guides#