gearpump-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Hyunseok Chang <>
Subject Regarding dynamic DAG
Date Mon, 12 Sep 2016 21:25:26 GMT

I'd like to know more about the dynamic DAG feature.

Let's say I have a DAG of:  source -> A -> B -> C.  I want to replace "B"
with "X" node in this chain.

How does node replacement happen internally?

Each processor consists of multiple parallel tasks, so node replacement
should involve killing multiple concurrent tasks for B, and somehow
introducing new tasks for X without affecting predecessor/successor tasks.
I'd like to know how this is done internally.

Also, can I change the parallelism (# of tasks) or type of partitioning
(hash <-> shuffle) of each processor dynamically at run time?


View raw message