hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ashutosh Chauhan (Commented) (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HIVE-2057) eliminate parser warning for "Identifier DOT Identifier"
Date Fri, 18 Nov 2011 19:48:53 GMT

    [ https://issues.apache.org/jira/browse/HIVE-2057?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13153083#comment-13153083
] 

Ashutosh Chauhan commented on HIVE-2057:
----------------------------------------

In recent builds, I sometimes get:
{{noformat}}
    [java] warning(209): /home/ashutosh/workspace/hive-2542/ql/src/java/org/apache/hadoop/hive/ql/parse/Hive.g:2283:1:
Multiple token rules can match input such as "'H'": KW_HAVING, KW_HOLD_DDLTIME, Identifier
     [java] As a result, tokens(s) Identifier,KW_HOLD_DDLTIME were disabled for that input
     [java] warning(209): /home/ashutosh/workspace/hive-2542/ql/src/java/org/apache/hadoop/hive/ql/parse/Hive.g:2222:1:
Multiple token rules can match input such as "'B'": KW_BY, KW_BOOLEAN, KW_BIGINT, KW_BUCKETS,
KW_BUCKET, KW_BEFORE, KW_BETWEEN, KW_BOTH, KW_BINARY, Identifier
     [java] As a result, tokens(s) Identifier,KW_BOTH,KW_BOOLEAN,KW_BINARY,KW_BIGINT,KW_BUCKET,KW_BETWEEN,KW_BEFORE,KW_BUCKETS
were disabled for that input
     [java] warning(209): /home/ashutosh/workspace/hive-2542/ql/src/java/org/apache/hadoop/hive/ql/parse/Hive.g:2170:1:
Multiple token rules can match input such as "'R'": KW_RIGHT, KW_REBUILD, KW_REPAIR, KW_RENAME,
KW_REDUCE, KW_ROW, KW_RCFILE, KW_READONLY, KW_REPLACE, KW_RLIKE, KW_REGEXP, KW_REVOKE, KW_READ,
KW_READS, KW_RANGE, KW_RECORDREADER, KW_RECORDWRITER, KW_RESTRICT, Identifier
     [java] As a result, tokens(s) KW_READONLY,KW_REGEXP,Identifier,KW_RENAME,KW_RECORDREADER,KW_RANGE,KW_RESTRICT,KW_REDUCE,KW_RCFILE,KW_RLIKE,KW_REVOKE,KW_ROW,KW_RECORDWRITER,KW_REPLACE,KW_READS,KW_REPAIR,KW_READ,KW_REBUILD
were disabled for that input
     [java] warning(209): /home/ashutosh/workspace/hive-2542/ql/src/java/org/apache/hadoop/hive/ql/parse/Hive.g:2150:1:
Multiple token rules can match input such as "'F'": KW_FALSE, KW_FROM, KW_FULL, KW_FUNCTIONS,
KW_FIRST, KW_FLOAT, KW_FORMAT, KW_FIELDS, KW_FILEFORMAT, KW_FUNCTION, KW_FORMATTED, KW_FETCH,
Identifier
     [java] As a result, tokens(s) Identifier,KW_FIELDS,KW_FORMATTED,KW_FUNCTION,KW_FETCH,KW_FUNCTIONS,KW_FILEFORMAT,KW_FORMAT,KW_FIRST,KW_FULL,KW_FLOAT,KW_FROM
were disabled for that input
     [java] warning(209): /home/ashutosh/workspace/hive-2542/ql/src/java/org/apache/hadoop/hive/ql/parse/Hive.g:2359:1:
Multiple token rules can match input such as "'!'": KW_NOT, NOTEQUAL
     [java] As a result, tokens(s) NOTEQUAL were disabled for that input
     [java] warning(209): /home/ashutosh/workspace/hive-2542/ql/src/java/org/apache/hadoop/hive/ql/parse/Hive.g:2277:1:
Multiple token rules can match input such as "'W'": KW_WHERE, KW_WITH, KW_WHEN, KW_WHILE,
Identifier
     [java] As a result, tokens(s) Identifier,KW_WITH,KW_WHILE,KW_WHEN were disabled for that
input
     [java] warning(209): /home/ashutosh/workspace/hive-2542/ql/src/java/org/apache/hadoop/hive/ql/parse/Hive.g:2428:1:
Multiple token rules can match input such as "'0'": CharSetLiteral, BigintLiteral, SmallintLiteral,
TinyintLiteral, Number, Identifier
     [java] As a result, tokens(s) Number,BigintLiteral,Identifier,TinyintLiteral,SmallintLiteral
were disabled for that input
     [java] warning(209): /home/ashutosh/workspace/hive-2542/ql/src/java/org/apache/hadoop/hive/ql/parse/Hive.g:2417:1:
Multiple token rules can match input such as "'"'": StringLiteral, CharSetLiteral
     [java] As a result, tokens(s) CharSetLiteral were disabled for that input
     [java] warning(209): /home/ashutosh/workspace/hive-2542/ql/src/java/org/apache/hadoop/hive/ql/parse/Hive.g:2443:1:
Multiple token rules can match input such as "'V'": KW_VIEW, Identifier
     [java] As a result, tokens(s) Identifier were disabled for that input
     [java] warning(209): /home/ashutosh/workspace/hive-2542/ql/src/java/org/apache/hadoop/hive/ql/parse/Hive.g:2300:1:
Multiple token rules can match input such as "'G'": KW_GROUP, KW_GRANT, Identifier
     [java] As a result, tokens(s) Identifier,KW_GRANT were disabled for that input
     [java] warning(209): /home/ashutosh/workspace/hive-2542/ql/src/java/org/apache/hadoop/hive/ql/parse/Hive.g:2314:1:
Multiple token rules can match input such as "'P'": KW_PRESERVE, KW_PARTITION, KW_PARTITIONS,
KW_PARTITIONED, KW_PERCENT, KW_PLUS, KW_PROCEDURE, KW_PURGE, Identifier
     [java] As a result, tokens(s) Identifier,KW_PLUS,KW_PARTITION,KW_PARTITIONS,KW_PARTITIONED,KW_PERCENT,KW_PROCEDURE,KW_PURGE
were disabled for that input
     [java] warning(209): /home/ashutosh/workspace/hive-2542/ql/src/java/org/apache/hadoop/hive/ql/parse/Hive.g:2287:1:
Multiple token rules can match input such as "'L'": KW_LIKE, KW_LEFT, KW_LOCAL, KW_LOAD, KW_LINES,
KW_LOCATION, KW_LIMIT, KW_LONG, KW_LOCK, KW_LOCKS, KW_LATERAL, Identifier
     [java] As a result, tokens(s) Identifier,KW_LOCK,KW_LOCAL,KW_LEFT,KW_LOCKS,KW_LATERAL,KW_LOCATION,KW_LOAD,KW_LINES,KW_LIMIT,KW_LONG
were disabled for that input
     [java] warning(209): /home/ashutosh/workspace/hive-2542/ql/src/java/org/apache/hadoop/hive/ql/parse/Hive.g:2200:1:
Multiple token rules can match input such as "'T'": KW_TRUE, KW_TABLE, KW_TABLES, KW_TRANSFORM,
KW_TO, KW_TINYINT, KW_TIMESTAMP, KW_TERMINATED, KW_TEXTFILE, KW_TABLESAMPLE, KW_TEMPORARY,
KW_TBLPROPERTIES, KW_THEN, KW_TRIGGER, KW_TOUCH, Identifier
     [java] As a result, tokens(s) KW_TIMESTAMP,KW_TRIGGER,KW_TEXTFILE,KW_TRANSFORM,KW_TERMINATED,Identifier,KW_TABLES,KW_TABLE,KW_TOUCH,KW_TABLESAMPLE,KW_TBLPROPERTIES,KW_TINYINT,KW_THEN,KW_TEMPORARY,KW_TO
were disabled for that input
     [java] warning(209): /home/ashutosh/workspace/hive-2542/ql/src/java/org/apache/hadoop/hive/ql/parse/Hive.g:2363:1:
Multiple token rules can match input such as "'>'": GREATERTHANOREQUALTO, GREATERTHAN
     [java] As a result, tokens(s) GREATERTHAN were disabled for that input
     [java] warning(208): /home/ashutosh/workspace/hive-2542/ql/src/java/org/apache/hadoop/hive/ql/parse/Hive.g:2457:1:
The following token definitions are unreachable: KW_AND,KW_ASC,KW_ORDER,KW_FROM,KW_AS,KW_DISTINCT,KW_INSERT,KW_OVERWRITE,KW_OUTER,KW_LEFT,KW_FULL,KW_ON,KW_PARTITION,KW_PARTITIONS,KW_TABLE,KW_TABLES,KW_INDEX,KW_INDEXES,KW_REBUILD,KW_FUNCTIONS,KW_SHOW,KW_REPAIR,KW_DIRECTORY,KW_LOCAL,KW_TRANSFORM,KW_USING,KW_DISTRIBUTE,KW_SORT,KW_UNION,KW_LOAD,KW_EXPORT,KW_IMPORT,KW_DATA,KW_INPATH,KW_IS,KW_NULL,KW_CREATE,KW_EXTERNAL,KW_ALTER,KW_CHANGE,KW_COLUMN,KW_FIRST,KW_AFTER,KW_DESCRIBE,KW_DROP,KW_RENAME,KW_TO,KW_COMMENT,KW_BOOLEAN,KW_TINYINT,KW_SMALLINT,KW_INT,KW_BIGINT,KW_FLOAT,KW_DOUBLE,KW_DATE,KW_DATETIME,KW_TIMESTAMP,KW_STRING,KW_ARRAY,KW_STRUCT,KW_MAP,KW_UNIONTYPE,KW_REDUCE,KW_PARTITIONED,KW_CLUSTERED,KW_SORTED,KW_INTO,KW_BUCKETS,KW_ROW,KW_FORMAT,KW_DELIMITED,KW_FIELDS,KW_TERMINATED,KW_ESCAPED,KW_COLLECTION,KW_ITEMS,KW_LINES,KW_STORED,KW_FILEFORMAT,KW_SEQUENCEFILE,KW_TEXTFILE,KW_RCFILE,KW_INPUTFORMAT,KW_OUTPUTFORMAT,KW_INPUTDRIVER,KW_OUTPUTDRIVER,KW_OFFLINE,KW_ENABLE,KW_DISABLE,KW_READONLY,KW_NO_DROP,KW_LOCATION,KW_TABLESAMPLE,KW_BUCKET,KW_OUT,KW_OF,KW_PERCENT,KW_CAST,KW_ADD,KW_REPLACE,KW_COLUMNS,KW_RLIKE,KW_REGEXP,KW_TEMPORARY,KW_FUNCTION,KW_EXPLAIN,KW_EXTENDED,KW_FORMATTED,KW_SERDE,KW_WITH,KW_DEFERRED,KW_SERDEPROPERTIES,KW_DBPROPERTIES,KW_LIMIT,KW_SET,KW_TBLPROPERTIES,KW_IDXPROPERTIES,KW_VALUE_TYPE,KW_ELEM_TYPE,KW_CASE,KW_WHEN,KW_THEN,KW_ELSE,KW_END,KW_MAPJOIN,KW_STREAMTABLE,KW_HOLD_DDLTIME,KW_CLUSTERSTATUS,KW_UTC,KW_UTCTIMESTAMP,KW_LONG,KW_DELETE,KW_PLUS,KW_MINUS,KW_FETCH,KW_INTERSECT,KW_IN,KW_DATABASE,KW_DATABASES,KW_MATERIALIZED,KW_SCHEMA,KW_SCHEMAS,KW_GRANT,KW_REVOKE,KW_SSL,KW_UNDO,KW_LOCK,KW_LOCKS,KW_UNLOCK,KW_SHARED,KW_EXCLUSIVE,KW_PROCEDURE,KW_UNSIGNED,KW_WHILE,KW_READ,KW_READS,KW_PURGE,KW_RANGE,KW_ANALYZE,KW_BEFORE,KW_BETWEEN,KW_BOTH,KW_BINARY,KW_CROSS,KW_CONTINUE,KW_CURSOR,KW_TRIGGER,KW_RECORDREADER,KW_RECORDWRITER,KW_SEMI,KW_LATERAL,KW_TOUCH,KW_ARCHIVE,KW_UNARCHIVE,KW_COMPUTE,KW_STATISTICS,KW_USE,KW_OPTION,KW_CONCATENATE,KW_SHOW_DATABASE,KW_UPDATE,KW_RESTRICT,KW_CASCADE,LESSTHANOREQUALTO,LESSTHAN,GREATERTHAN,DIV,DOLLAR,SmallintLiteral,TinyintLiteral,Number,COMMENT


{{noformat}}

which looks to me like inconsistent behavior from antlr. Pig dealt with similar antlr vagaries
in PIG-2055 at which point they decided to upgrade to 3.4 to get rid of inconsistent antlr
behavior. We should consider to upgrade antlr version too.
                
> eliminate parser warning for "Identifier DOT Identifier"
> --------------------------------------------------------
>
>                 Key: HIVE-2057
>                 URL: https://issues.apache.org/jira/browse/HIVE-2057
>             Project: Hive
>          Issue Type: Improvement
>          Components: Diagnosability, Query Processor
>            Reporter: John Sichi
>
> I noticed this warning in recent builds:
> {noformat}
> build-grammar:
>      [echo] Building Grammar /data/users/jsichi/open/hive-trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/Hive.g
 ....
>      [java] ANTLR Parser Generator  Version 3.0.1 (August 13, 2007)  1989-2007
>      [java] warning(200): /data/users/jsichi/open/hive-trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/Hive.g:1503:5:
Decision can match input such as "Identifier DOT Identifier" using multiple alternatives:
1, 2
>      [java] As a result, alternative(s) 2 were disabled for that input
> {noformat}
> This was introduced by HIVE-1517.  Is there a way to get rid of it?

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

       

Mime
View raw message