apex-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Sunil Parmar <spar...@threatmetrix.com>
Subject Occasional Out of order tuples when emitting from a thread
Date Tue, 21 Feb 2017 18:05:30 GMT
Hi there,
We have the following setup:

  *   we have a generic operator that's processing tuples in its input port
  *   in the input port's process method, we check for a condition, and:
     *   if the condition is met, the tuple is emitted to the next operator right away (in
the process method)
     *   Otherwise, if the condition is not met, we store the tuple  in some cache and we
use some threads that periodically check the condition to become true. Once the condition
is true, the threads call the emit method on the stored tuples.

With this setup, we occasionally encounter the following error:
2017-02-15 17:29:09,364 ERROR com.datatorrent.stram.engine.GenericNode: Catastrophic Error:
Out of sequence BEGIN_WINDOW tuple 58a4046100003b7f on port transformedJSON while expecting
58a4046100003b7e

Is there a way to make the above work correctly?
If not, can you recommend a better way of doing this?
How can we ensure window assignment is done synchronously before emitting tuples ?

Thanks very much in advance...
-allan

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message