hadoop-hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ashish Thusoo (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HIVE-142) Create a metastore check command
Date Fri, 09 Jan 2009 23:05:59 GMT

    [ https://issues.apache.org/jira/browse/HIVE-142?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12662552#action_12662552
] 

Ashish Thusoo commented on HIVE-142:
------------------------------------

Ok. found what is happening..

This is not actually backtracking related but actually the related to the normal code that
antlr generates. If you make the createFunction branch the last branch in ddlStatement i.e.

ddlStatement
    : createStatement
    | dropStatement
    | alterStatement
    | descStatement
    | showStatement
    | metastoreCheck
    | createFunctionStatement
    ;

instead of

ddlStatement
    : createStatement
    | dropStatement
    | alterStatement
    | descStatement
    | showStatement
    | createFunctionStatement
    | metastoreCheck
    ;

Basically the first and the createStatement and createFunctionStatement branches need a lookahead
for disambiguation as both of them start with the create key word. For some reason antlr generates
if (pred_for_create) {
  take alternative 1;
}
if (true) {
  take alternative 2;
}

incase these one of these branches is the last statement (I think this has got to do with
the fact that there are no more branches to check if createFunctionStatement is that last
branch).

otherwise,
if (pred_for_create) {
  take alternative 1;
}
if (pred_for_create_function) {
  take alternative 2;
}

That changes the error.

I agree with Prasad, that both the errors are equally useless, so +1 to capturing the new
log.


> Create a metastore check command
> --------------------------------
>
>                 Key: HIVE-142
>                 URL: https://issues.apache.org/jira/browse/HIVE-142
>             Project: Hadoop Hive
>          Issue Type: New Feature
>          Components: Metastore
>            Reporter: Johan Oskarsson
>            Assignee: Johan Oskarsson
>             Fix For: 0.2.0
>
>         Attachments: HIVE-142.patch, HIVE-142.patch, HIVE-142.patch, HIVE-142.patch,
HIVE-142.patch
>
>
> We need a command to verify that the information in the metastore reflects the data that
is on hdfs. For example partitions can be deleted on hdfs but still be in the metastore.
> From Joydeep Sen Sarma, see ticket HIVE-126 for the full comment:
> for a command line interface - one might want to check the entire database or just a
table or even just one partition. other metadata checks will also be added over time (for
example - do the file types on disk agree with metadata records, bucketing information etc).
So, here's a strawman proposal for a new command:
> alter table <DB>[.TABLE [PARTITION-SPEC]] check [TYPE-LIST]
> where TYPE by default is 'all' (check for all kinds of errors), but can be specified
to a specific type. For example - in this case - we can have a type called 'partitions' (and
then over time we can add other types like 'fileformat' etc.). for v1 - we can just drop the
type-list altogether.
> the check command can produce a list of things that need to be done to fix the format
(like adding any directories not in the metastore - but in hdfs - to the metastore). actually
performing of such steps would require a user confirmation (y/n).

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message