apex-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Sanjay Pujare <san...@datatorrent.com>
Subject Re: can operators emit on a different from the operator itself thread?
Date Wed, 10 Aug 2016 21:36:49 GMT
If the goal is to do this validation through static analysis of operator code, I guess it is
possible but is going to be non-trivial. And there could be false positives and false negatives.

Also I suppose this discussion applies to processor operators (those having both in and out
ports) so Ram’s example of JdbcPollInputOperator may not be applicable here?

On 8/10/16, 2:04 PM, "Ashwin Chandra Putta" <ashwinchandrap@gmail.com> wrote:

    In a separate thread I mean.
    
    Regards,
    Ashwin.
    
    On Wed, Aug 10, 2016 at 2:01 PM, Ashwin Chandra Putta <
    ashwinchandrap@gmail.com> wrote:
    
    > + dev@apex.apache.org
    > - users@apex.apache.org
    >
    > This is one of those best practices that we learn by experience during
    > operator development. It will save a lot of time during operator
    > development if we can catch and throw validation error when someone emits
    > tuples in a non separate thread.
    >
    > Regards,
    > Ashwin
    >
    > On Wed, Aug 10, 2016 at 1:57 PM, Munagala Ramanath <ram@datatorrent.com>
    > wrote:
    >
    >> For cases where use of a different thread is needed, it can write tuples
    >> to a queue from where the operator thread pulls them --
    >> JdbcPollInputOperator in Malhar has an example.
    >>
    >> Ram
    >>
    >> On Wed, Aug 10, 2016 at 1:50 PM, hsy541@gmail.com <hsy541@gmail.com>
    >> wrote:
    >>
    >>> Hey Vlad,
    >>>
    >>> Thanks for bringing this up. Is there an easy way to detect unexpected
    >>> use of emit method without hurt the performance. Or at least if we can
    >>> detect this in debug mode.
    >>>
    >>> Regards,
    >>> Siyuan
    >>>
    >>> On Wed, Aug 10, 2016 at 11:27 AM, Vlad Rozov <v.rozov@datatorrent.com>
    >>> wrote:
    >>>
    >>>> The short answer is no, creating worker thread to emit tuples is not
    >>>> supported by Apex and will lead to an undefined behavior. Operators in
Apex
    >>>> have strong thread affinity and all interaction with the platform must
    >>>> happen on the operator thread.
    >>>>
    >>>> Vlad
    >>>>
    >>>
    >>>
    >>
    >
    >
    > --
    >
    > Regards,
    > Ashwin.
    >
    
    
    
    -- 
    
    Regards,
    Ashwin.
    



Mime
View raw message