accumulo-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Mike Drob <mad...@cloudera.com>
Subject Re: Q4A Project
Date Tue, 28 Apr 2015 01:38:05 GMT
Andrew,

This is a cool thing to work on, I hope you have great success!

A couple of questions about the motivations behind this, if you don't mind -
- There are several SQL implementations already in the Hadoop ecosystem. In
what ways do you expect this to improve upon
Hive/Impala/Phoenix/Presto/Spark SQL? I haven't looked at the code, so it
is quite possible you're already using one of those technologies.
- In a conversation with some HP engineers earlier this year, they
mentioned that building a SQL-92 layer is the easy part, and that a mature
optimization engine is the really hard part. This is where Oracle may still
be leaps and bounds ahead of its nearest competitors. Do you have plans for
a query planner? If not, you might be back to writing MapReduce jobs sooner
than you think.

Look forward to seeing more!

Mike

On Mon, Apr 27, 2015 at 7:37 PM, Andrew Wells <awells@clearedgeit.com>
wrote:

> I have been working on a project, tentatively called Q4A (Query for
> Accumulo). Another possible name is ASQ (Accumulo Streaming Query) [discus].
>
> This is a streaming query as the query is completed via a stream, should
> never group data in memory. To batch, intermediate results would be written
> back to Accumulo temporarily.
>
>
> The *primary goal* is to have a complete SQL implementation native to
> Accumulo.
>
> *Why do this?*
> I am getting tired of writing bad java code to query a database. I would
> rather write bad SQL code. Also, people should be able to get queries out
> faster and it shouldn't take a developer.
>
>
> *Native To Accumulo*:
>
>    - There should be no special format to read a database created by Q4A
>    - There should be no special format for Q4A to query a table
>    - All tables are tables available to Q4A
>    - Any special tables, are stored away from the users databases
>    (indexes, column definitions, etc)
>
> *Other Goals*:
>
>    - Implement the entire SQL definition (currently all of SQLite)
>    - Create JDBC Driver/Server
>    - Push down Expressions to the Tablet Servers
>    - Install-less queries, use Q4A jar directly against any Accumulo
>    Cluster ( less push-down expressions)
>    - documentation :o
>    - testing ;)
>
> *Does it work?*
> Not yet, the project is still a work in progress. and I will be working on
> it at the Accumulo Summit this year. Progress is slow as I am getting
> married in about a month and some change.
>
> *Questions:*
> If you have questions about Q4A as here, I will be at the Accumulo Summit
> @ ClearEdgeIT Table and Hackathon.
>
> *WHERE IS TEH LINK?!1!*
> Oh.... here: https://github.com/agwells0714/q4a
>
> --
> *Andrew George Wells*
> *Software Engineer*
> *awells@clearedgeit.com <awells@clearedgeit.com>*
>
>

Mime
View raw message