Databricks Pitches Spark Real-Time Mode Against Flink
Start here
Databricks argues Apache Spark should be the default engine for most teams, saying its Real-Time Mode brings Spark streaming to millisecond end-to-end latency and removes the main reason teams added Apache Flink. In Databricks' own benchmark at 10,000 records per second, Spark Real-Time Mode matched Flink on stateless work at 3 ms p99 and was faster on stateful aggregation (14 ms vs 45 ms) and stream-static joins (9 ms vs 95 ms). Databricks still names cases where Flink fits: very large keyed state, complex event processing and event-time timers. Adopting Flink in those cases usually means running a second engine alongside Spark. The benchmark figures come from Databricks and have not been independently verified.
Generated from reports · updated 16 min ago
Report timeline
Follow the reports to see the event from different sides.
- Databricks BlogApache Spark vs Apache Flink: which should you use?
Databricks argues Apache Spark is the better default for most teams, since Real-Time Mode brings Spark streaming to millisecond end-to-end latency and removes the main reason teams added Flink. In Databricks' published benchmark at 10,000 records per second, Spark Real-Time Mode matched Flink on stateless work (3 ms p99) and beat it on stateful aggregation (14 ms vs 45 ms) and stream-static joins (9 ms vs 95 ms). Flink remains a fit for very large keyed state, complex event processing and event-time timers, but adopting it usually means operating a second engine alongside Spark.
Heat for this event
Not enough continuous observations yet to draw a trend.