apex-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From David Yan <david...@gmail.com>
Subject Re: [DISCUSS] Custom Control Tuples Design
Date Fri, 16 Dec 2016 16:57:53 GMT
 // DefaultInputPort
  public void processControl(ControlTuple tuple)
  {
    // Default Implementation to avoid need to implement it in all
implementations
  }
{code}

{code}
 // DefaultOutputPort
  public void emitControl(ControlTuple tuple)
  {
  }

I think we don't need to expose the ControlTuple class to the operator
developers because the window ID is just the current window ID when these
methods are called. How about making them just Object? We also need to
provide the way for the user to specify custom serializer for the control
tuple.

David


On Thu, Dec 15, 2016 at 12:43 AM, Bhupesh Chawda <bhupesh@datatorrent.com>
wrote:

> Hi All,
>
> Here are the initial interfaces:
>
> {code}
>  // DefaultInputPort
>   public void processControl(ControlTuple tuple)
>   {
>     // Default Implementation to avoid need to implement it in all
> implementations
>   }
> {code}
>
> {code}
>  // DefaultOutputPort
>   public void emitControl(ControlTuple tuple)
>   {
>   }
> {code}
>
> We have an option to add these methods to the interfaces - InputPort and
> OutputPort; But these would not be backward compatible and also not
> consistent with the current implementation of basic data tuple flow (as
> with process() and emit()).
>
> We also need to expose an interface / class for users to wrap their object
> and emit downstream. This should be part of API.
>
> {code}
> public class ControlTuple extends Tuple
> {
>   Object userObject;
>
>   public ControlTuple(long windowId, Object userObject)
>   {
>     //
>   }
> }
> {code}
>
> The emitted tuples would traverse the same flow as with other control
> tuples. The plan is to intercept the control tuples in GenericNode and use
> the Reservior to emit the control tuples at the end of the window.
>
> GenericNode seems to be the best place to buffer incoming custom control
> tuples without delivering them immediately to the operator port. Once the
> end of the window is reached, we plan to use the reservoir sink to push
> them to the port. This is different behavior than any other control tuple
> where we are changing the order of tuples in the stream. The custom control
> tuples will be buffered and not delivered to the ports until the end of the
> window.
> To accomplish this, we need to have a public method in SweepableReservoir
> which allows to put a tuple back in the sink of the reservoir.
>
> ~ Bhupesh
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message