March 7 - June 27, 2024 - Nstream is hitting the road with Confluent on the #DataInMotionTour! / Learn More

Overview

Nstream offers various libraries that facilitate consuming and processing data from external sources.

Recall from the Toolkit Overview that ingestion is the conversion of externally-hosted data into state that may drive actions taken by Web Agents. Ingestion can be further divided into four steps:

  1. Connecting to an external process
  2. Receiving data (which may be serialized or encoded) from that process
  3. Transforming that data into a server-side object – either a plain old Java object (POJO), or an instance of swim.structure.Value
  4. Performing logic within Web Agents that may be driven by the contents of the object.

For each of the libraries that are documented in this Ingress Bridge section:

Due to this separation of responsibilities, it is entirely expected that one may require multiple libraries to handle a single data source. For example, the “schema” variation of the Confluent Cloud Tutorial uses both nstream.adapter.confluent to pull data from hosted Kafka topics and nstream.adapter.avro to seamlessly handle Avro deserialization.

* POJO-oriented Step 3 is best accomplished using pure Java. Step 4 is accomplished using either the Recon DSL or pure Java.


Nstream is licensed under the Redis Source Available License 2.0 (RSALv2).