apex-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Amol Kekre <a...@datatorrent.com>
Subject Re: Why does emit require current thread to be the operator thread?
Date Mon, 10 Apr 2017 19:45:48 GMT
This constraint was introduced as allowing two threads to emit data creates
lots of bad situations
1. The emit is triggered between end_window and begin_window. This was a
critical blocker
2. Order no longer guaranteed, upon replay getting wrong order of events
within a window. This was something to worry about, but not a blocker

We had users report this problem.

The solution is to pass the data to main thread and have the main thread
emit this data during one of start-window, process, end-window calls.
Ideally during start-window or end-window so as to guarantee order. Keeping
this code in start or end window also ensures that process call remains


E:amol@datatorrent.com | M: 510-449-2606 | Twitter: @*amolhkekre*


On Mon, Apr 10, 2017 at 12:39 PM, Ganelin, Ilya <Ilya.Ganelin@capitalone.com
> wrote:

> Hello – I’ve got an operator that runs a cleanup thread (separate from the
> main event loop) and triggers a callback when an item is removed from an
> internal data structure. I would like for this callback to emit data from
> one of the operator’s ports, but I run into the following Exception:
> (From DefaultOutputPort.java, line 58)
> if (operatorThread != null && Thread.*currentThread*() != operatorThread)
> {
>   // only under certain modes: enforce this
>   throw new IllegalStateException("Current thread " + Thread.
> *currentThread*().getName() +
>       " is different from the operator thread " +
> operatorThread.getName());
> }
> I could obviously extend DefaultOperatorPort to bypass this but I’d like
> to understand why that constraint is there and if there’s a good way to
> work around it.
> Would love to hear the community’s thoughts. Thanks!
> - Ilya Ganelin
> [image: id:image001.png@01D1F7A4.F3D42980]
> ------------------------------
> The information contained in this e-mail is confidential and/or
> proprietary to Capital One and/or its affiliates and may only be used
> solely in performance of work or services for Capital One. The information
> transmitted herewith is intended only for use by the individual or entity
> to which it is addressed. If the reader of this message is not the intended
> recipient, you are hereby notified that any review, retransmission,
> dissemination, distribution, copying or other use of, or taking of any
> action in reliance upon this information is strictly prohibited. If you
> have received this communication in error, please contact the sender and
> delete the material from your computer.

  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message