Web Agents: The Future of Digital Twins and Streaming Data
Web Agents are the key to fulfilling the promise of digital twins: real-time observability at scale to drive smarter decision-making. With a history dating back to 1960s space exploration, the digital twin idea — using a virtual model of a physical object or system to better comprehend the present and predict the future — has been around for a while.
Because most digital twins simply mirror the state of the entities they represent and don’t dynamically relate to each other, we often see references to “the potential of digital twins.” For many, digital twins have remained a concept that is useful in theory but hasn’t been executed in a way that delivers true value. Even in today’s streaming data era, it seems almost impossible to create digital twins that operate in real time based on the processing power needed to handle the continuous flow of billions of data records generated by real-world events…
… until now. Web Agents offer a way to build digital twins from real-time data and to execute business logic to drive decision automation. In other words, Web Agents not only make full fidelity digital twins possible, but they also help them evolve from passive to active tools.
How do Web Agents take digital twins to the next level?
From Web Agents’ “analyze-then-store” architecture to their ability to be woven together into a real-time web, there are several ways in which these distributed objects enable smarter, faster digital twins.
First, let’s examine why the capabilities of conventional digital twins fall short of what most businesses desire.
The problem with traditional digital twins
Dr. Michael Grieves, Executive Director of the Digital Twin Institute, was one of the first people to articulate the digital twin model and its core underlying elements: real space, virtual space, the link for data flow from real space to virtual space, and the link for information flow from virtual space to real space and virtual subspaces.
The problem with traditional digital twins lies in those information “links.” Previously there was no way to keep information flowing in real time, as is required for many business use cases. Companies looking to leverage digital twin streaming data would often save it to a data lake or other data store and then perform analysis.
This “store-then-analyze” approach greatly increases latency, which is a major problem when you need to take action on time-sensitive matters. For example, a company needs to be notified as soon as possible if there is a mechanical issue with a delivery drone to prevent delays and keep customers satisfied.
Web Agents and the “analyze-then store” model
Web Agents, on the other hand, are able to compute data as soon as things change in the real world. These computations can range from simple transformations and aggregations all the way to advanced analysis and machine learning. They power contextual data analytics and real-time visualizations based on massive amounts of streaming data to create robust digital twins without lag.
Using Web Agents as digital twins of real-world entities transcends big data and streaming data silos, replacing the limitations of batch-based processing and latency-prone database queries with an “analyze-then-store” model in which data is stored only if it is useful.
Why Web Agents provide superior business value
With traditional cloud analytics, databases are queried to process data, update state, and derive insights. This approach works well when data rates below network speeds are tolerable, volume is relatively inexpensive to store, and/or decision timeframes take longer than processing delays.
However, today’s business world is seeing a rapidly increasing number of scenarios where the ability to derive real-time insights from massive volumes of streaming data is required to inform critical business decisions. With the traditional “store-then-analyze” approach mentioned above in the drone example, data is not getting where it’s needed in time.
In the “analyze-then-store” paradigm powered by Web Agents, incoming live data is processed first and only stored if required (e.g., compliance purposes) or desired (e.g., create a complete picture of a customer’s journey and predict future behavior). Because Web Agents are optimized for continuous, correlated analysis of constantly changing data, they provide the ideal combination of speed and situational awareness.
Real-time observability via streaming APIs
Streaming APIs are the key to achieving real-time observability at scale. Web Agents stream their state to interested clients (observers) via streaming APIs. Each Web Agent has a URI address, like a REST endpoint. However, unlike RESTful Web Services, Web Agents are stateful due to streaming APIs that instantly detect changes in streaming data. Traditional digital twins lack real-time observability because they don’t have streaming built into their architecture.
Real-time observability enabled by streaming APIs gives companies the ability to make critical business decisions faster and with more meaningful context, helping them unlock the full potential of their streaming data. In practice, this could entail proactively reaching out to a customer based on recent events (e.g., sending an SMS coupon based on a store they’ve visited) or automatically starting a job in the field (e.g., dispatching a technician to quickly address a faulty machine component).
The incredible connectivity of Web Agents
When examining Web Agents and the ideal digital twin, one feature that should be underscored is their ability to be dynamically woven together. Millions of Web Agents, each representing a real-world entity (e.g., a vehicle, a fulfillment center), can be linked in a real-time web capable of processing both streaming and stationary data to model complex, large-scale systems.
Linking Web Agents allows them to see each other’s current state, in real time, as well as express real-world relationships, such as containment and proximity. This is how Web Agents use data to build sophisticated models of complex, large-scale systems as they change — cutting-edge digital twins.
Web Agents are the evolution of digital twins because they are able to:
- Process their own data
- Compute state changes
- Execute complex business logic
- Bypass database round-trip delays
- React dynamically to their real-world data sources
- Deliver responses in real-time
What are the key characteristics of a Web Agent?
Web Agents unify state, compute, persistence, and messaging into a single, vertically integrated data processing architecture capable of handling the complexity and flexibility required for a large scale distributed system.
State Web Agents preserve their data and compute context locally between operations—aka maintain statefulness. They retain everything they need to do their jobs as state, including data like recent history, real-time status of related entities, and partial computation products for rapid re-evaluation of analytics.
Compute Web Agents run general-purpose compute processes, including tasks such as executing context-sensitive business logic in response to real-world events and communicating with external message brokers and database engines to feed Web Agents with data.
Persistence Modern streaming workloads are very latency-sensitive. Web Agents are able to reduce latency by an order of magnitude because they retain data in memory for data locality, keeping persistence off the critical performance path. Persisting data locally allows Web Agents to scale millions of granular read-modify-write operations per second.
Messaging Rather than send sequential messages, Web Agents communicate with each other by observing changes to their state in a cache-coherent manner. This allows them to dynamically optimize the utilization of the network to reduce congestion and avoid buffer bloat—in other words, operate fast and keep latency low.
Get started with Web Agents for free
Through their interconnectivity and ability to immediately deliver insights on large quantities of real-time streaming data, Web Agents fulfill the promise of digital twins. Web Agents enable stateful microservices and serve as endpoints for streaming APIs, which in turn power real-time user interfaces that interact with entity models and business logic to visualize key insights. Interested in trying out Web Agents for your digital twin use case? Check out SwimOS to get up and running with an open-source implementation of Web Agents today.