drill-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Abdel Hakim Deneche <adene...@maprtech.com>
Subject Re: Why limit operator keeps calling next() even after the limit is reached ?
Date Thu, 29 Oct 2015 19:00:57 GMT
Thanks Steven, I will look into RawBatchBuffer.

On Thu, Oct 29, 2015 at 11:52 AM, Steven Phillips <steven@dremio.com> wrote:

> I believe kill() will only stop the upstream fragments from sending
> batches, but it does nothing about the batches that have already been sent.
> When kill() is called on the RawBatchBuffer, this will release all of the
> batches in the queue. But I believe it is still necessary to wait for all
> remaining batches to arrive, so they can be cleared. It's possible that
> it's not necessary to do this in LimitRecordBatch, and that we are handling
> this in the RawBatchBuffer. You would have to examine the code to confirm.
>
> On Thu, Oct 29, 2015 at 11:38 AM, Abdel Hakim Deneche <
> adeneche@maprtech.com
> > wrote:
>
> > Hey all,
> >
> > As part of DRILL-991 when LimitRecordBatch receives enough records it
> calls
> > kill() on it's upstream to inform the remaining operators and fragments
> > that they can stop sending batches.
> >
> > But, limit operator will keep calling next() until it gets a NONE. Is
> there
> > a specific reason for this behavior ?
> >
> > Thanks
> >
> > --
> >
> > Abdelhakim Deneche
> >
> > Software Engineer
> >
> >   <http://www.mapr.com/>
> >
> >
> > Now Available - Free Hadoop On-Demand Training
> > <
> >
> http://www.mapr.com/training?utm_source=Email&utm_medium=Signature&utm_campaign=Free%20available
> > >
> >
>



-- 

Abdelhakim Deneche

Software Engineer

  <http://www.mapr.com/>


Now Available - Free Hadoop On-Demand Training
<http://www.mapr.com/training?utm_source=Email&utm_medium=Signature&utm_campaign=Free%20available>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message