Blog Posts

Fault tolerance in MillWheels

Fault tolerance in MillWheels

Introduction Fault tolerance is super important for modern streaming systems since data is always in transit and failure can happen at any stage of the system. We can run into issues like lost messages, processing the same message twice, or keeping track of state properly. This can lead to unreliable results or even system downtime. For example, think about a system that monitors web searches to spot sudden trends. If a part of the system fails or some data gets dropped, it might miss a new trend or, even worse, trigger false alerts.

Read More
Gossip Protocol

Gossip Protocol

Gossip Introduction to gossip protocol Why do we need gossip protocol ? Centralized Coordination Overhead

Read More