apex-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Sanjay Pujare <san...@datatorrent.com>
Subject Re: [Proposal] DAG listener
Date Wed, 10 Aug 2016 17:40:11 GMT
Looks like a good idea but would like more details about the use case (“example ... to access
the operator name while using Batched StatsListener”).

Do you propose very fine grained subscription model? Also what “major activities in the
DAG” will be available thru this listener?



On 8/10/16, 8:20 AM, "Sandesh Hegde" <sandesh@datatorrent.com> wrote:

    Any operators can subscribe to Stram Events affecting the DAG.
    
    Implementation will most probably use heartbeat.
    
    On Tue, Aug 9, 2016 at 11:43 PM Tushar Gosavi <tushar@datatorrent.com>
    wrote:
    
    > Hi Sandesh,
    >
    > Dag changes are handled in Stram, and operators are running in
    > different containers.
    > Are you suggesting an RPC interface or operator request for sending
    > this information
    > from Stram to all partitions of the interested operator?
    >
    > - Tushar.
    >
    >
    >
    > On Wed, Aug 10, 2016 at 11:28 AM, Sandesh Hegde <sandesh@datatorrent.com>
    > wrote:
    > > Hi All,
    > >
    > > As we add more features to support batch use cases, there will be a need
    > to
    > > access more information about the DAG from an operator. One example is
    > the need
    > > to access the operator name while using Batched StatsListener.
    > >
    > > The idea here is to implement DAG Listener ( similar to StatsListener ),
    > > which can potentially give every information present in the Stram.
    > > Operators will also have visibility into all the major activities in the
    > > DAG.
    > >
    > > Thoughts?
    > >
    > > Thanks
    >
    



Mime
View raw message