apex-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Sanjay Pujare <san...@datatorrent.com>
Subject Re: Visitor API for DAG
Date Thu, 17 Nov 2016 18:05:35 GMT
There is a risk if the user written code blocks the thread or crashes the process. What are
the real life examples of this use case?


On 11/17/16, 9:21 AM, "amol kekre" <amolhkekre@gmail.com> wrote:

    +1. Opening up the API for users to put in their own code is good. In
    general we should enable users to register their code in a lot of scenerios.
    
    Thks
    Amol
    
    On Thu, Nov 17, 2016 at 9:06 AM, Tushar Gosavi <tushar@datatorrent.com>
    wrote:
    
    > Yes, It could happen after current DAG validation and before the
    > application master is launched.
    >
    > - Tushar.
    >
    >
    > On Thu, Nov 17, 2016 at 8:32 PM, Munagala Ramanath <ram@datatorrent.com>
    > wrote:
    > > When would the visits happen ? Just before normal validation ?
    > >
    > > Ram
    > >
    > > On Wed, Nov 16, 2016 at 9:50 PM, Tushar Gosavi <tushar@apache.org>
    > wrote:
    > >
    > >> Hi All,
    > >>
    > >> How about adding visitor like API for DAG in Apex, and an api to
    > >> register visitor for the DAG.
    > >> Possible use cases are
    > >> -  Validator visitor which could validate the dag
    > >> -  Visitor to inject properties/attribute in the operator/streams from
    > >> some external sources.
    > >> -  Platform does not support validation of individual operators.
    > >> developer could write a validator visitor which would call validate
    > >> function of operator if it implements Validator interface.
    > >> - generate output schema based on operator config and input schema,
    > >> and set the schema on output stream.
    > >>
    > >> Sample API :
    > >>
    > >> dag.registerVisitor(DAGVisitor visitor);
    > >>
    > >> Call order of visitorFunctions.
    > >> - preVisitDAG(Attributes) // dag attributes
    > >>   for all operators
    > >>   - visitOperator(OperatorMeta meta) // access to operator, name,
    > >> attributes, properties
    > >>  ports
    > >>   - visitStream(StreamMeta meta) // access to
    > >> stream/name/attributes/properties/ports
    > >> - postVisitDAG()
    > >>
    > >> Regards,
    > >> -Tushar.
    > >>
    >
    



Mime
View raw message