asterixdb-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Taewoo Kim <wangs...@gmail.com>
Subject Re: LIMIT push down doesn't really LIMIT the source op.
Date Wed, 30 Dec 2015 19:14:18 GMT
I'm sorry that I missed this posting. Given our PUSH based architecture, I
am not sure LIMIT operator can "finalize/stop" frames from its input side.
To remedy this issue, we are working on LIMIT pushdown, which passes LIMIT
information to an index-search or external-sort so that only certain number
of tuples will be generated from the source.

Best,
Taewoo

On Wed, Dec 30, 2015 at 11:09 AM, Chen Li <chenli@gmail.com> wrote:

> Maybe Jianfeng and Taewoo can provide some input?
>
> On Sun, Dec 27, 2015 at 5:41 AM, Wail Alkowaileet <wael.y.k@gmail.com>
> wrote:
> > Hi all,
> >
> > I was working on a large external dataset and when I query something with
> > LIMIT clause, I see that the job waits for the
> > ExternalDataScanOperatorDescriptor to finish scanning the whole file.
> >
> > When I looked at the plan, I found that the limit operator described
> > in StreamLimitRuntimeFactory.
> > The implementation only skips the frames when the limit expression
> reached
> > and it doesn't *finalize/stop* the source operator from pushing frames,
> > which halts the overall job.
> >
> >             public void nextFrame(ByteBuffer buffer) throws
> > HyracksDataException {
> >                 if (afterLastTuple) {
> >                     return; //Accepting frames without appending it to
> the
> > parent operator
> >                 }
> >
> > I think that applied for any source operator.
> >
> >
> >
> > --
> >
> > *Regards,*
> > Wail Alkowaileet
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message