drill-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Aman Sinha (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (DRILL-3929) Support the ability to query database tables using external indices
Date Tue, 10 Nov 2015 18:12:11 GMT

    [ https://issues.apache.org/jira/browse/DRILL-3929?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14999045#comment-14999045

Aman Sinha commented on DRILL-3929:

Thanks for your feedback [~julianhyde] and [~jamestaylor].  Let me respond to Julian's feedback
first.  I think it really comes down to requirement.  

Are we trying to build an optimizer/execution model for generalized index/MV selection  where
suppose the same column may appear in multiple indexes/MVs and the goal is to find the best
match based on costing ?  RDBMs's have done a lot of work on this  (a good paper that combines
both MV and Indexes is: http://www.vldb.org/conf/2000/P496.pdf).  This assumes appropriate
statistics exist.  

Or, are we trying to leverage the full power of external indices such as Elasticsearch etc.
in which users may create an index offline (i.e without Drill's involvement) but then use
Drill to query the table (not the index directly).  I was more focused on this scenario. 
Here, the choice is just two:  full table scan or index lookup followed by join-back to table,
since the index-specific optimization is left upto Elasticsearch.  I understand your concern
that this won't meet the previously mentioned requirement.  I will discuss this with some
more folks but my I think my proposal should not preclude from trying to satisfy the first
requirement in the future. 

> Support the ability to query database tables using external indices           
> ------------------------------------------------------------------------------
>                 Key: DRILL-3929
>                 URL: https://issues.apache.org/jira/browse/DRILL-3929
>             Project: Apache Drill
>          Issue Type: New Feature
>          Components: Execution - Relational Operators, Query Planning & Optimization
>            Reporter: Aman Sinha
>            Assignee: Aman Sinha
> This is a placeholder for adding support in Drill to query database tables using external
indices.  I will add more details about the use case and a preliminary design proposal.  

This message was sent by Atlassian JIRA

View raw message