hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Brock Noland (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HIVE-1511) Hive plan serialization is slow
Date Mon, 19 Aug 2013 13:19:51 GMT

    [ https://issues.apache.org/jira/browse/HIVE-1511?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13743799#comment-13743799
] 

Brock Noland commented on HIVE-1511:
------------------------------------

Two example errors I found below. 

{noformat}
org.apache.hadoop.hive.ql.parse.SemanticException: Generate Map Join Task Error: Class cannot
be created (missing no-arg constructor): org.antlr.runtime.CommonToken
Serialization trace:
token (org.apache.hadoop.hive.ql.parse.ASTNode)
children (org.apache.hadoop.hive.ql.parse.ASTNode)
expressionMap (org.apache.hadoop.hive.ql.parse.RowResolver)
rr (org.apache.hadoop.hive.ql.parse.OpParseContext)
opParseCtxMap (org.apache.hadoop.hive.ql.plan.MapWork)
mapWork (org.apache.hadoop.hive.ql.plan.MapredWork)
  at org.apache.hadoop.hive.ql.optimizer.physical.CommonJoinTaskDispatcher.processCurrentTask(CommonJoinTaskDispatcher.java:532)
  at org.apache.hadoop.hive.ql.optimizer.physical.AbstractJoinTaskDispatcher.dispatch(AbstractJoinTaskDispatcher.java:182)
  at org.apache.hadoop.hive.ql.lib.TaskGraphWalker.dispatch(TaskGraphWalker.java:111)
  at org.apache.hadoop.hive.ql.lib.TaskGraphWalker.walk(TaskGraphWalker.java:194)
  at org.apache.hadoop.hive.ql.lib.TaskGraphWalker.startWalking(TaskGraphWalker.java:139)
  at org.apache.hadoop.hive.ql.optimizer.physical.CommonJoinResolver.resolve(CommonJoinResolver.java:79)
  at org.apache.hadoop.hive.ql.optimizer.physical.PhysicalOptimizer.optimize(PhysicalOptimizer.java:90)
  at org.apache.hadoop.hive.ql.parse.MapReduceCompiler.compile(MapReduceCompiler.java:292)
  at org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.analyzeInternal(SemanticAnalyzer.java:8338)
  at org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:278)
{noformat}


{noformat}
2013-08-16 17:38:11,297 ERROR ql.Driver (SessionState.java:printError(416)) - FAILED: ArrayIndexOutOfBoundsException
1
java.lang.ArrayIndexOutOfBoundsException: 1
  at org.apache.hadoop.hive.ql.metadata.Partition.getBucketPath(Partition.java:427)
  at org.apache.hadoop.hive.ql.optimizer.SamplePruner.prune(SamplePruner.java:199)
  at org.apache.hadoop.hive.ql.optimizer.GenMapRedUtils.setTaskPlan(GenMapRedUtils.java:558)
  at org.apache.hadoop.hive.ql.optimizer.GenMapRedUtils.setTaskPlan(GenMapRedUtils.java:398)
  at org.apache.hadoop.hive.ql.optimizer.GenMRFileSink1.processFS(GenMRFileSink1.java:715)
  at org.apache.hadoop.hive.ql.optimizer.GenMRFileSink1.process(GenMRFileSink1.java:160)
  at org.apache.hadoop.hive.ql.lib.DefaultRuleDispatcher.dispatch(DefaultRuleDispatcher.java:90)
  at org.apache.hadoop.hive.ql.lib.DefaultGraphWalker.dispatchAndReturn(DefaultGraphWalker.java:94)
  at org.apache.hadoop.hive.ql.parse.GenMapRedWalker.walk(GenMapRedWalker.java:54)
  at org.apache.hadoop.hive.ql.parse.GenMapRedWalker.walk(GenMapRedWalker.java:65)
  at org.apache.hadoop.hive.ql.parse.GenMapRedWalker.walk(GenMapRedWalker.java:65)
  at org.apache.hadoop.hive.ql.parse.GenMapRedWalker.walk(GenMapRedWalker.java:65)
  at org.apache.hadoop.hive.ql.lib.DefaultGraphWalker.startWalking(DefaultGraphWalker.java:109)
  at org.apache.hadoop.hive.ql.parse.MapReduceCompiler.compile(MapReduceCompiler.java:259)
  at org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.analyzeInternal(SemanticAnalyzer.java:8338)
  at org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:278)
  at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:437)
  at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:341)
{noformat}
                
> Hive plan serialization is slow
> -------------------------------
>
>                 Key: HIVE-1511
>                 URL: https://issues.apache.org/jira/browse/HIVE-1511
>             Project: Hive
>          Issue Type: Improvement
>    Affects Versions: 0.7.0
>            Reporter: Ning Zhang
>            Assignee: Mohammad Kamrul Islam
>         Attachments: HIVE-1511.4.patch, HIVE-1511.5.patch, HIVE-1511.patch, HIVE-1511-wip2.patch,
HIVE-1511-wip3.patch, HIVE-1511-wip4.patch, HIVE-1511-wip.patch
>
>
> As reported by Edward Capriolo:
> For reference I did this as a test case....
> SELECT * FROM src where
> key=0 OR key=0 OR key=0 OR  key=0 OR key=0 OR key=0 OR key=0 OR key=0
> OR key=0 OR key=0 OR key=0 OR
> key=0 OR key=0 OR key=0 OR  key=0 OR key=0 OR key=0 OR key=0 OR key=0
> OR key=0 OR key=0 OR key=0 OR
> ...(100 more of these)
> No OOM but I gave up after the test case did not go anywhere for about
> 2 minutes.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message