March 7 - June 27, 2024 - Nstream is hitting the road with Confluent on the #DataInMotionTour! / Learn More

Toolkit Lane Design

The Swim platform provides lanes to expose APIs to and store fields of web agents. Ideas and best practices on how to organize these lanes can be found here. This guide will expand on these ideas and show practically how the Nstream Toolkit can be used to create and organize lanes. The toolkit provides configurable lanes in the form of patches, see the patches guide.

Value Lanes

The Nstream Toolkit provides a standard packaged value lane in the form of a LatestValuePatch.

The patch exposes two lanes:

As the name suggests the patch should be used to store the latest value of a data set. Ingress patches will commonly command addEvent lanes directly, this will keep all agents up-to-date with the most recent event for each entity from that datasource. Value lanes will often be one-to-one with datasources for each entity.

  @node {
    pattern: "/agent/:id"
    @agent(class: "nstream.adapter.common.patches.LatestValuePatch")
  }

Map Lanes

Timeseries

The Nstream Toolkit provides a standard packaged timeseries map lane in the form of a HistoryPatch. This patch is very similar to LatestValuePatch, storing multiple events instead of only the most recent.

The patch exposes two lanes:

As the name suggest the patch should be used to store a recent history of a data set that is timestamped. Ingress patches will commonly command addEvent lanes directly, this will keep all agents up-to-date with a recent history of events for each entity. Map Lanes will often be one-to-one with timestamped datasources for each entity.

  @node {
    pattern: "/agent/:id"
    @agent(class: "nstream.adapter.common.patches.HistoryPatch") {
      maxHistorySize: 20
      extractEpochMillisFromEvent: $timestamp
    }
  }

Join Value Lanes

The Nstream Toolkit provides a standard packaged join value lane in the form of a GroupPatch, used in conjunction with MemberPatch to populate the lane.

For a more detailed guide on how to use these patches, see the agent grouping guide.

Two lanes are exposed:

As the name suggests the patches should be used to group agents, creating an aggregated map of all the member agents. The lane of the entity the join value lane downlinks can be any lane but works well with the LatestValuePatch, giving the aggregate agent the current state of all it’s members.

Ingress patches will commonly command addEvent lanes directly, this will keep all agents grouped correctly. Member patches will often be one-to-one with datasources where the event contains some kind of field that can be used to group the entities.

A group field is not necessarily needed in every event if the group is static across all the entity agents - useful for seeing aggregations across all entities of a certain type.

  @node {
    uri: "/aggregate"
    @agent(class: "nstream.adapter.common.patches.GroupPatch")
  }
  
  @node {
    pattern: "/agent/:id"
    @agent(class: "nstream.adapter.common.patches.LatestValuePatch")
    @agent(class: "nstream.adapter.common.patches.MemberPatch") {
      groupUri: "/aggergate"
    }

Nstream is licensed under the Redis Source Available License 2.0 (RSALv2).