hadoop-hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ashutosh Chauhan (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HIVE-1546) Ability to plug custom Semantic Analyzers for Hive Grammar
Date Tue, 14 Sep 2010 16:44:35 GMT

    [ https://issues.apache.org/jira/browse/HIVE-1546?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12909318#action_12909318

Ashutosh Chauhan commented on HIVE-1546:


Continuing from discussion yesterday w.r.t new interface/hooks that you were suggesting. As
far as I understood you have two primary concerns:
 * You do not want to have config variable enabled SemanticAnalyzer factory producing different
Analyzers. Rather, you want to have only one  HiveSemanticAnalyzer.
 * You also dont want others to extend SemanticAnalyzer, DDLSemanticAnalyzer and other Hive

As a solution to this, there will be new hooks introduced. These pre and post hooks will be
called by Hive at different times during Semantic Analysis phase. To me, it seems there are
few potential problems: 
 * Hive will need to call these hooks from each and every method of its Analyzer which opens
up whole lot more integration points to external tools (like Howl) and burden on those Analyzers
then the current approach of factory enabled instantiation and extending of those classes.
 * Either there will be a one generic hook or many different hooks. If there is only one generic
hook, then passed in information also need to be generic and as a result it will be a root
of AST Node. If so, in any particular hook I am using I need to traverse the whole AST to
get the node I am interested in.     
 * If the hooks are more specific, then more specific information can be passed in which is
more useful, but then you will have long list of such hooks.
 * Even if these concerns are taken care of, it seems that all of Howl's requirements may
not get satisfied (like getting hold of DDLTask and modifying it, doing metadata operations
etc.) I can elaborate on them further once I can see what you are proposing.

Clearly from my description, you can see that I didnt get what you were envisioning and how
is it better then current design. I must be missing something. Would you like to propose what
you were suggesting? The attached patch and Howl classes on this jira should be sufficient
to spec out Howl's requirements, if you need them.  

> Ability to plug custom Semantic Analyzers for Hive Grammar
> ----------------------------------------------------------
>                 Key: HIVE-1546
>                 URL: https://issues.apache.org/jira/browse/HIVE-1546
>             Project: Hadoop Hive
>          Issue Type: Improvement
>          Components: Metastore
>    Affects Versions: 0.7.0
>            Reporter: Ashutosh Chauhan
>            Assignee: Ashutosh Chauhan
>             Fix For: 0.7.0
>         Attachments: hive-1546-3.patch, hive-1546-4.patch, hive-1546.patch, hive-1546_2.patch,
> It will be useful if Semantic Analysis phase is made pluggable such that other projects
can do custom analysis of hive queries before doing metastore operations on them. 

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message