apex-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Pramod Immaneni <pra...@datatorrent.com>
Subject Broader ability for plan changes in STRAM
Date Mon, 10 Apr 2017 15:13:48 GMT
Currently, DAG topological changes are limited to repartitioning of
existing operators after stats update and client directed logical plan
changes via web services. We have seen the need for more dynamic plan
changes especially from batch use cases and other scenarios.

I wanted to start a discussion around broadening this by increasing the
scope of changes that can be done after say a stats update from only
repartitioning and also allow more scenarios in which the changes can be
done.

After stats update for example, instead of only having the ability to
trigger a repartition we could give the ability for the user code to change
the plan. Plan changes can also be triggered after major events such as
when a stram event gets created. In batch for example after shutdown is
received from one stage another plan can be created and started.

Thanks

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message