Overview
Nstream offers various libraries that facilitate consuming and processing data from external sources.
Recall from the Toolkit Overview that ingestion is the conversion of externally-hosted data into state that may drive actions taken by Web Agents. Ingestion can be further divided into four steps:
- Connecting to an external process
- Receiving data (which may be serialized or encoded) from that process
- Transforming that data into a server-side object – either a plain old Java object (POJO), or an instance of
swim.structure.Value
- Relaying that object to downstream, Entity Web Agents for further processing
For each of the libraries that are documented in this Ingress Bridge section:
- If the article’s title ends in Ingress, then its corresponding library emphasizes Steps 1 and 2
- If the article’s title ends in Handling, then its corresponding library emphasizes
Value
-oriented Step 3*.
Due to this separation of responsibilities, it is entirely expected that one may require multiple libraries to handle a single data source.
For example, the “schema” variation of the Confluent Cloud Tutorial uses both nstream.adapter.confluent
to pull data from hosted Kafka topics and nstream.adapter.avro
to seamlessly handle Avro deserialization.
* POJO-oriented Step 3 is best accomplished using pure Java. Step 4 is accomplished using either the Recon DSL or pure Java.
Nstream is licensed under the Redis Source Available License 2.0 (RSALv2).