hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jitendra Nath Pandey (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (HIVE-6496) Queries fail to Vectorize.
Date Thu, 27 Feb 2014 06:40:20 GMT

     [ https://issues.apache.org/jira/browse/HIVE-6496?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Jitendra Nath Pandey updated HIVE-6496:
---------------------------------------

    Description: 
Following issues are causing many queries to fail to vectorize:
1) NPE because row resolver is null.
2) VectorUDFAdapter doesn't handle decimal.
3) Decimal cast to boolean, timestamp, string fail because classes are not annotated appropriately.
4) Decimal modulo fails to vectorize because GenericUDFOPMod is not annotated.


  was:
In some queries, we see NPE in Vectorizer because row resolver is not available in parse context.

{code}
FAILED: NullPointerException null
2014-02-21 14:03:02,277 ERROR ql.Driver: FAILED: NullPointerException null
java.lang.NullPointerException
at org.apache.hadoop.hive.ql.optimizer.physical.Vectorizer.getVectorizationContext(Vectorizer.java:728)
at org.apache.hadoop.hive.ql.optimizer.physical.Vectorizer.access$100(Vectorizer.java:125)
at org.apache.hadoop.hive.ql.optimizer.physical.Vectorizer$VectorizationNodeProcessor.process(Vectorizer.java:428)
at org.apache.hadoop.hive.ql.lib.DefaultRuleDispatcher.dispatch(DefaultRuleDispatcher.java:90)
at org.apache.hadoop.hive.ql.lib.DefaultGraphWalker.dispatchAndReturn(DefaultGraphWalker.java:94)
at org.apache.hadoop.hive.ql.lib.DefaultGraphWalker.dispatch(DefaultGraphWalker.java:78)
at org.apache.hadoop.hive.ql.lib.PreOrderWalker.walk(PreOrderWalker.java:54)
at org.apache.hadoop.hive.ql.lib.DefaultGraphWalker.startWalking(DefaultGraphWalker.java:109)
at org.apache.hadoop.hive.ql.optimizer.physical.Vectorizer$VectorizationDispatcher.vectorizeMapWork(Vectorizer.java:344)
at org.apache.hadoop.hive.ql.optimizer.physical.Vectorizer$VectorizationDispatcher.convertMapWork(Vectorizer.java:289)
at org.apache.hadoop.hive.ql.optimizer.physical.Vectorizer$VectorizationDispatcher.dispatch(Vectorizer.java:279)
at org.apache.hadoop.hive.ql.lib.TaskGraphWalker.dispatch(TaskGraphWalker.java:111)
at org.apache.hadoop.hive.ql.lib.TaskGraphWalker.walk(TaskGraphWalker.java:194)
at org.apache.hadoop.hive.ql.lib.TaskGraphWalker.startWalking(TaskGraphWalker.java:139)
at org.apache.hadoop.hive.ql.optimizer.physical.Vectorizer.resolve(Vectorizer.java:523)
at org.apache.hadoop.hive.ql.parse.TezCompiler.optimizeTaskPlan(TezCompiler.java:252)
at org.apache.hadoop.hive.ql.parse.TaskCompiler.compile(TaskCompiler.java:216)
at org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.analyzeInternal(SemanticAnalyzer.java:9303)
at org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:327)
at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:445)
at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:345)
at org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:1078)
at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1121)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1014)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1004)
at org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:268)
at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:220)
at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:424)
at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:359)
at org.apache.hadoop.hive.cli.CliDriver.processReader(CliDriver.java:457)
at org.apache.hadoop.hive.cli.CliDriver.processFile(CliDriver.java:467)
at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:125)
at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:424)
at org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:793)
at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:687)
at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:626)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.apache.hadoop.util.RunJar.main(RunJar.java:212)
{code}


> Queries fail to Vectorize.
> --------------------------
>
>                 Key: HIVE-6496
>                 URL: https://issues.apache.org/jira/browse/HIVE-6496
>             Project: Hive
>          Issue Type: Sub-task
>            Reporter: Jitendra Nath Pandey
>            Assignee: Jitendra Nath Pandey
>         Attachments: HIVE-6496.1.patch, HIVE-6496.2.patch
>
>
> Following issues are causing many queries to fail to vectorize:
> 1) NPE because row resolver is null.
> 2) VectorUDFAdapter doesn't handle decimal.
> 3) Decimal cast to boolean, timestamp, string fail because classes are not annotated
appropriately.
> 4) Decimal modulo fails to vectorize because GenericUDFOPMod is not annotated.



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

Mime
View raw message