apex-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Chetan Narsude <che...@datatorrent.com>
Subject Re: Throttling of `emitTuples`?
Date Tue, 01 Sep 2015 00:10:00 GMT
Hi Brennon,

  The emitTuples is called as many times as possible within the given
application window. Most efficiency is achieved when a single invocation of
emitTuples takes streaming_window_width time. Often it's not possible for
emitTuples to predict how much time it's going to take and hence the
platform gives it a hint that it can output more events if there is still
time in given streaming window.

  If operator has nothing to emit, it could just return the emitTuples
call. When emitTuples returns without emitting anything, the platform
automatically throttles the calls to emitTuples (to slow down) and
optionally invokes IdleTimeHandler.

  So it's left upto operator to throttle its own behavior. Generally most
operators will output events in batches of a few thousand at a time or try
to empty up their queue if it's being built asynchronously.

--
Chetan

On Mon, Aug 31, 2015 at 4:25 PM, York, Brennon <Brennon.York@capitalone.com>
wrote:

> Hey all, is there a property out there that throttles the `emitTuples`
> call for input operators? I’ve been hunting down various properties and
> can’t seem to find it for the life of me. I’m sure I’m missing something
> simple…
> ________________________________________________________
>
> The information contained in this e-mail is confidential and/or
> proprietary to Capital One and/or its affiliates and may only be used
> solely in performance of work or services for Capital One. The information
> transmitted herewith is intended only for use by the individual or entity
> to which it is addressed. If the reader of this message is not the intended
> recipient, you are hereby notified that any review, retransmission,
> dissemination, distribution, copying or other use of, or taking of any
> action in reliance upon this information is strictly prohibited. If you
> have received this communication in error, please contact the sender and
> delete the material from your computer.
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message