apex-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Amol Kekre <a...@datatorrent.com>
Subject Re: can operators emit on a different from the operator itself thread?
Date Wed, 10 Aug 2016 21:42:08 GMT
Sanjay just reminded me of my typo -> I meant between end_window and
start_window :)

Thks
Amol

On Wed, Aug 10, 2016 at 2:36 PM, Sanjay Pujare <sanjay@datatorrent.com>
wrote:

> If the goal is to do this validation through static analysis of operator
> code, I guess it is possible but is going to be non-trivial. And there
> could be false positives and false negatives.
>
> Also I suppose this discussion applies to processor operators (those
> having both in and out ports) so Ram’s example of JdbcPollInputOperator may
> not be applicable here?
>
> On 8/10/16, 2:04 PM, "Ashwin Chandra Putta" <ashwinchandrap@gmail.com>
> wrote:
>
>     In a separate thread I mean.
>
>     Regards,
>     Ashwin.
>
>     On Wed, Aug 10, 2016 at 2:01 PM, Ashwin Chandra Putta <
>     ashwinchandrap@gmail.com> wrote:
>
>     > + dev@apex.apache.org
>     > - users@apex.apache.org
>     >
>     > This is one of those best practices that we learn by experience
> during
>     > operator development. It will save a lot of time during operator
>     > development if we can catch and throw validation error when someone
> emits
>     > tuples in a non separate thread.
>     >
>     > Regards,
>     > Ashwin
>     >
>     > On Wed, Aug 10, 2016 at 1:57 PM, Munagala Ramanath <
> ram@datatorrent.com>
>     > wrote:
>     >
>     >> For cases where use of a different thread is needed, it can write
> tuples
>     >> to a queue from where the operator thread pulls them --
>     >> JdbcPollInputOperator in Malhar has an example.
>     >>
>     >> Ram
>     >>
>     >> On Wed, Aug 10, 2016 at 1:50 PM, hsy541@gmail.com <hsy541@gmail.com
> >
>     >> wrote:
>     >>
>     >>> Hey Vlad,
>     >>>
>     >>> Thanks for bringing this up. Is there an easy way to detect
> unexpected
>     >>> use of emit method without hurt the performance. Or at least if we
> can
>     >>> detect this in debug mode.
>     >>>
>     >>> Regards,
>     >>> Siyuan
>     >>>
>     >>> On Wed, Aug 10, 2016 at 11:27 AM, Vlad Rozov <
> v.rozov@datatorrent.com>
>     >>> wrote:
>     >>>
>     >>>> The short answer is no, creating worker thread to emit tuples is
> not
>     >>>> supported by Apex and will lead to an undefined behavior.
> Operators in Apex
>     >>>> have strong thread affinity and all interaction with the platform
> must
>     >>>> happen on the operator thread.
>     >>>>
>     >>>> Vlad
>     >>>>
>     >>>
>     >>>
>     >>
>     >
>     >
>     > --
>     >
>     > Regards,
>     > Ashwin.
>     >
>
>
>
>     --
>
>     Regards,
>     Ashwin.
>
>
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message