asterixdb-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Wail Alkowaileet <>
Subject LIMIT push down doesn't really LIMIT the source op.
Date Sun, 27 Dec 2015 13:41:50 GMT
Hi all,

I was working on a large external dataset and when I query something with
LIMIT clause, I see that the job waits for the
ExternalDataScanOperatorDescriptor to finish scanning the whole file.

When I looked at the plan, I found that the limit operator described
in StreamLimitRuntimeFactory.
The implementation only skips the frames when the limit expression reached
and it doesn't *finalize/stop* the source operator from pushing frames,
which halts the overall job.

            public void nextFrame(ByteBuffer buffer) throws
HyracksDataException {
                if (afterLastTuple) {
                    return; //Accepting frames without appending it to the
parent operator

I think that applied for any source operator.


Wail Alkowaileet

  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message