asterixdb-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From abdullah alamoudi <bamou...@gmail.com>
Subject Re: Modified/Custom plan: Push-down SELECT for external source.
Date Sun, 28 Aug 2016 07:49:02 GMT
Would be really nice to have that. I had plans to do it at some points but
didn't get a chance to pursue them.
Not only it is useful for Parquet, we can also use it with other formats.
we will need a new rewrite rule that pushes projection to datasources. This
should be generic enough to be used with internal datasets as well
(assuming we support PAX at some point).

I am sure there are many hidden obstacle to writing this rewrite rule but I
think that it can be done in a week.

Cheers,
Abdullah.

On Sat, Aug 27, 2016 at 11:37 PM, Mike Carey <dtabass@gmail.com> wrote:

> It would be INCREDIBLY helpful be able to do this (also for internal
> sources) - e.g., even our basic index access operators could optionally
> accept a filter to apply to the records they'd otherwise simply return.
> This could especially useful when combined with limit pushdown, I would
> think, too.
>
>
> On 8/26/16 3:44 PM, Wail Alkowaileet wrote:
>
>> Hi AsterixDBers.
>>
>> Is there any easy way to push-down filter to an external source (in my
>> case
>> Parquet) without being too intrusive ?
>>
>> This can perform way faster than STREAM-SELECT, as parquet can potentially
>> skip Row Groups while scanning.
>>
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message