Apps vs. Analytics
What problem are you trying to solve? A data problem? Or a business problem?
If it’s a data problem, you need analytics. If it’s a business problem: you need an application. It’s usually a mix of both.
Analytics model data
Analytics interpret data using a mathematical model.
Applications model businesses
Analytics interpret the meaning of data and analytics according to a model of the business.
Analytics are passive
Analytics are pure mathematical functions: given some input data, they produce an output.
Analytics have nothing to say about what to do with the output.
In order to function correctly, analytics are required to mot have “side effects”. A side effect is any external influence on the world that isn’t captured by the mathematical equation.
A side effect could be: sending an email, restarting a device, making an API call, or updating a dashboard. Basically any useful action you can think of is a pesky side effect when viewed from an analytics perspective.
Applications are active
The whole point of applications, on the other hand, is to have “side effects”. Given some data, an application males some impact on the world, either through a user, or through autonomous action.
Desegregating analytics and applications
Most enterprises today have separate analytics and applications teams.
At first glance, this makes sense. They’re two different development disciplines.
Analytics have traditionally served long-term strategy and planning, while applications focus on day-to-day operations.
But as the time horizon of analytics get closer and closer, the two worlds are increasingly colliding.
Applications need analytics
Better analytics enable more intelligent, informed, and actionable business logic.
Analytics need applications
Applications “realize” the value of analytics. Without applications to close the loop on a business problem, analytics can become seen as science projects that aren’t delivering on their promises.
Analytics platforms can’t run business logic
Because of the whole “side effect” restriction.
Stateless application platforms are limited in the analytics they can compute
Big data analytics don’t map onto the remote procedure calls that make up the core functionality of stateless applications. Big data needs to “shuffle” data across parallel processing nodes in order to scale.
Application platforms can be used as rudimentary event processors. But their stateless nature makes it exceedingly expensive to combine streaming data with context.
Streaming databases are trying to empower event-driven microservices by offloading stream enrichment. But that doesn’t solve the problem because, the app is still stuck transforming events. Event processors make for an extremely weak and limited application architecture.
Need a stateful application architecture than can do both
There’s no fundamental reason that analytics and applications have to be so difficult to reconcile. The key is to have an entity-parallel stateful architecture. The “parallel” part of “entity-parallel” let’s you scale the compute needed to perform analytics. And the “entity” part of “entity-parallel” let’s you run business logic on business entities.