drill-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From rahul challapalli <challapallira...@gmail.com>
Subject Re: Lucene Plugin :: Join Filter and pushdown
Date Thu, 14 Jan 2016 23:08:47 GMT
Use Case : In the case of a left join between a non-index table and a
lucene index, it is more efficient to read the join keys from the non-index
table and push them into the LuceneGroupScan. This way we can avoid reading
the whole index.
I was suggesting converting the plan for Q1 into a plan similar to Q2 using
an optimizer rule.
  Q1.) select * from tbl1 left join tbl2 on tbl1.col1 = tbl2.col1
  Q2.) select * from tbl1 left join (select * from tbl2 where tbl2.col1 in
(select col1 from tbl1))

Any other suggestions or pointers are appreciated

- Rahul


On Thu, Jan 14, 2016 at 2:52 PM, Stefán Baxter <stefan@activitystream.com>
wrote:

> Hi,
>
> I'm working on the Lucene plugin (see previous email) and the focus now is
> support for joins with filter push-down to avoid the default table scan
> that is provided by default.
>
> I'm fairly new to Drill and in over my head, to be honest, but this is fun
> and with this addition the Lucene plugin could be come usable or at least
> worth exploring.
>
> Is there anyone here that could assist me a bit?
>
> Current status:
>
>    - The lucene plugin is working and join filters are partially
>    - RelOptRuleOperand is constructed and DrillJoinRel.conditions are
>    processed by a sceleton class (The "normal" queries are already being
>    processed fairly well)
>
> There are probably more things involved then I can imagine at this point
> and perhaps I'm naive in thinking someone has the time to assist a relative
> noob on such a task but examples are also appreciated. The plugins that I
> have seen seem to have relatively no join-filter logic so a rich
> example/blueprint would also be great.
>
> Regards,
>  -Stefán
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message