heron-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Thomas Cooper <tom.n.coo...@gmail.com>
Subject Stateful Updates
Date Sat, 31 Mar 2018 01:08:51 GMT
Hi all,

After chatting to the Heron Team at Twitter and Microsoft I have written up
a proposal for giving Heron the ability to preserve state across a topology
update (change in component parallelism). This is problem for components
with incoming fields based connections as keys will not hash to the same
task ID after an update. I have defined the problem (as I see it) and
propsoed two solutions in the google doc below:

https://docs.google.com/document/d/1X0pS9uwevn16nYYqrGjEvWQMWQLwti7Q0hmW6lmgyk8/edit?usp=sharing

I would love to get feedback on these proposals. Feel free to blow holes in
my assumptions and correct any misinterpretations I have made!

Thanks,

Tom Cooper
W: www.tomcooper.org.uk  | Twitter: @tomncooper
<https://twitter.com/tomncooper>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message