incubator-drill-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ananth Gundabattula <agundabatt...@threatmetrix.com>
Subject Re: Help regarding new storage plugin and Query parser documentation
Date Mon, 24 Nov 2014 19:52:02 GMT
Hello Carol,

 > Which Systems 
We are having a graph storage format on top of cassandra and an indexing
system running as a set of impala tables. We would want to direct the
query that can best a fit a given sub-system and then expose the result to
the external client as a simple jdbc result set . In the above example, we
fire the query on Impala, get back row keys/Vertex Ids and then serve full
rows if requested as the result from the graph/cassandra layer.

>drill already does this for dfs, hive, hbase, mongo
By analyzing the query I mean, the indexing system might have a ³reduced²
format of the data wherein the actual data is replaced by entries in the
lookup table. This is saving us a lot of space and better caching
strategies in the first place. Hence we would like to ³rewrite² the
incoming query before firing it off into the Impala sub-system and extract
the result and use it further to process the query. Hence the request for
plug-ability of a custom query parser.

I understand the above use cases are entirely specific for our project
needs and hence was wondering if we can alter Drill to the needs we have.

Regards,
Ananth 

On 25/11/2014 2:14 am, "Carol McDonald" <cmcdonald@maprtech.com> wrote:

>> mix of many systems
>which systems?
>>analyzing the query on the fly and route it to the best sub-system
>drill already does this for dfs, hive, hbase, mongo
>
>On Sun, Nov 23, 2014 at 4:47 PM, Ananth Gundabattula <
>agundabattula@threatmetrix.com> wrote:
>
>> Hello All,
>>
>> I am trying to experiment with Drill a bit for a project and have the
>> following two questions :
>>
>> 1. Is there any documentation how to generate a storage plugin for
>>Drill ?
>>
>> 2. Is there any documentation as to how plugin a custom query parser ? -
>> Basically the Data warehouse we are building is a mix of many systems
>>and a
>> given query can be answered by more than one subsystem. However we would
>> like the query to be answered by analyzing the query on the fly and
>>route
>> it to the best sub-system that can answer the query. Drill seems to be
>>an
>> excellent option as it also provides a jDBC driver option. However on
>> looking at the Drill wiki, I could not find any document which gives an
>> idea as to how to plugin both a new Storage engine and a new Query
>>parser
>> that meets criteria for ³rewriting an incoming query²
>>
>> Thanks a lot for your time.
>>
>> Regards,
>> Ananth
>>


Mime
View raw message