apex-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (APEXCORE-3) Ability for an operator to populate DAG at launch time
Date Mon, 21 Dec 2015 22:57:46 GMT

    [ https://issues.apache.org/jira/browse/APEXCORE-3?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15067208#comment-15067208

ASF GitHub Bot commented on APEXCORE-3:

Github user tushargosavi commented on the pull request:

    I have made changes in my https://github.com/tushargosavi/incubator-apex-core/tree/pull189
 branch. I will have to do force push in your branch because checkstyle changes are worked
into original commits. Is it fine? 

> Ability for an operator to populate DAG at launch time
> ------------------------------------------------------
>                 Key: APEXCORE-3
>                 URL: https://issues.apache.org/jira/browse/APEXCORE-3
>             Project: Apache Apex Core
>          Issue Type: New Feature
>            Reporter: Amol Kekre
>            Assignee: Vlad Rozov
>            Priority: Critical
> Apex should have an operator API that lets the operator generate DAG during launch time.
This will mean the following
> - Logical DAG will have one operator. This is the operator that will generate a DAG underneath
> - Physical plan will have the DAG generated by the operator
> - Execution plan will mimic physical plan + container location etc.
> For example lets say we have three operators in a DAG (app) A->B->C
> B during launch time generates a DAG B1->B2->B3, then the physical plan will be
> A->B1->B2->B3->C
> This should work irrespective of number of ports, etc. A typical flattening. The operators
inside of B (B1, B2, B3) should have properties and attributes just as any. Users should be
able to access these at run time and compile time. B itself should support properties and
attributes that B1, B2, B3 can inherit from.
> This is a very critical feature as it will open up users to plug-in their own engines
and still take up complete operability support from Apex engine.

This message was sent by Atlassian JIRA

View raw message