heron-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ning Wang <wangnin...@gmail.com>
Subject Re: Fault tolerance
Date Tue, 20 Mar 2018 23:40:24 GMT
Two features I can think of:

Dhalion
https://blog.acolyer.org/2017/06/30/dhalion-self-regulating-stream-processing-in-heron/

Backpressure
Streammanager is responsible for all data transactions and it monitors the
data consumption. If a process runs slow and incoming data buffer is full,
it triggers backpressure to stop new data until the data in pipeline is
consumed.



On Tue, Mar 20, 2018 at 10:26 AM, Thuvarakan Tharmarajasingam <
thuva4.dev@gmail.com> wrote:

> Hello,
> I am currently working on a distributed fault tolerance system for big data
> toolkit. Could you please give me an overview of how Heron achieved the
> fault tolerance?
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message