Overview

Nstream offers various libraries that facilitate consuming and processing data from external sources.

Recall from the Toolkit Overview that ingestion is the conversion of externally-hosted data into state that may drive actions taken by Web Agents. Ingestion can be further divided into four steps:

1. Connecting to an external process
2. Receiving data (which may be serialized or encoded) from that process
3. Transforming that data into a server-side object – either a plain old Java object (POJO), or an instance of swim.structure.Value
4. Relaying that object to downstream, Entity Web Agents for further processing

For each of the libraries that are documented in this Ingress Bridge section:

• If the article’s title ends in Ingress, then its corresponding library emphasizes Steps 1 and 2
• If the article’s title ends in Handling, then its corresponding library emphasizes Value-oriented Step 3*.

Due to this separation of responsibilities, it is entirely expected that one may require multiple libraries to handle a single data source. For example, the “schema” variation of the Confluent Cloud Tutorial uses both nstream.adapter.confluent to pull data from hosted Kafka topics and nstream.adapter.avro to seamlessly handle Avro deserialization.

* POJO-oriented Step 3 is best accomplished using pure Java. Step 4 is accomplished using either the Recon DSL or pure Java.