apex-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Amol Kekre <a...@datatorrent.com>
Subject Re: Support for dynamic topology
Date Tue, 30 Aug 2016 18:46:50 GMT
Hyunseok,
The new route in the pipeline will have a new Z operator. If you want to
use the old Z operator (state?) then things get tricky. Do confirm that you
do not plan to use old Z operator.

Thks,
Amol


On Tue, Aug 30, 2016 at 11:02 AM, Sandesh Hegde <sandesh@datatorrent.com>
wrote:

> Hello hs,
>
> Yes, you can change the topology from the Apex CLI.
>
> One possible sequence of commands for your scenario is described below,
>
> connect appid
> begin-logical-plan-change
> create-operator ....
> add-stream-sink ...  ( for the input of P )
> add-stream-sink ... ( for the output of P )
> remove-operator ...
> submit
>
> Note: All the required operators needs to be in the package.
>
> Thanks
>
>
>
>
>
> On Tue, Aug 30, 2016 at 7:22 AM Hyunseok Chang <hyunseok.chang@gmail.com>
> wrote:
>
>> Hi,
>>
>> I'd like to know more about Apex support for dynamic topology.
>>
>> From my readings on Apex, I understand we can add additional parallel
>> tasks for each operator and change data partitioning among them dynamically
>> at run time (so-called data partitioning and unification features).
>>
>> My question is can we change the "logical" DAG at run time?
>>
>> Let's say my logical DAG is a chain of three operators X, Y & Z (i.e., X
>> -> Y -> Z).  Now at run time I want to replace operator Y with operator P,
>> such that the new logical DAG would look like X -> P -> Z.
>>
>> Is it something I can do with Apex?
>>
>> Thanks!
>> -hs
>>
>>

Mime
View raw message