hadoop-hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Neil Conway (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HIVE-385) Split Driver.run() into compile() + exec()
Date Mon, 06 Apr 2009 01:50:12 GMT

    [ https://issues.apache.org/jira/browse/HIVE-385?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12695952#action_12695952

Neil Conway commented on HIVE-385:

Yeah, using multiple Drivers might be the easiest in the short term. I think in the long term,
it might make sense to have an interface like:

* QueryPlan: the AST + output of the query optimizer, immutable
* QueryResult: the output of applying the query executor to a QueryPlan; provides the Driver's
GetResults() method.
* The driver just becomes a function that takes an (immutable) QueryPlan and produces a QueryResult,
which contains all the state associated with the query execution

But for now, the candidate patch I attached just separates the existing code in Driver into
two methods, compile() and execute(). Review/feedback welcome.

One potential cause of error is that the order of some operations has changed -- for example,
the old code sets HiveConf.ConfVars.HIVEQUERYSTRING, sets a QueryId and does SessionState.get().getHiveHistory().startQuery()
before doing parse+analyze, whereas we now do parse+analyze first. AFAICS that shouldn't break
anything, but let me know if that's incorrect.

> Split Driver.run() into compile() + exec()
> ------------------------------------------
>                 Key: HIVE-385
>                 URL: https://issues.apache.org/jira/browse/HIVE-385
>             Project: Hadoop Hive
>          Issue Type: Improvement
>          Components: Query Processor
>            Reporter: Neil Conway
>            Assignee: Neil Conway
>         Attachments: hive_driver_compile.patch
> Currently, Driver.run() parses, analyzes, and then executes a given query string. This
isn't very flexible: it would be nice to be able to parse and analyze a query to get a query
plan, and then execute that query plan at a later time (zero or more times). This would also
be a necessary prerequisite for supporting server-side prepared statements.
> I'd like to change Driver as follows:
> * Add a new method Driver.compile(String), which parses and analyzes the given query
string, and returns an object representing the planned query
> * Add a new method Driver.execute(Plan), which runs the given planned query
> * Reimplement run(String) as { p = compile(String); execute(p); }
> Comments welcome; I should have a candidate patch ready soon.

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message