hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Chao (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (HIVE-7591) GenMapRedUtils::addStatsTask only assumes either MapredWork or TezWork
Date Fri, 01 Aug 2014 15:44:39 GMT

     [ https://issues.apache.org/jira/browse/HIVE-7591?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Chao updated HIVE-7591:
-----------------------

    Attachment: HIVE-7591-spark.patch

A quick fix for this issue. I think a better solution might be throwing exception
when seeing an unexpected work type, but at the moment it involves propagating the exception
across multiple calls, which I'm not sure if it's the right thing to do.

> GenMapRedUtils::addStatsTask only assumes either MapredWork or TezWork
> ----------------------------------------------------------------------
>
>                 Key: HIVE-7591
>                 URL: https://issues.apache.org/jira/browse/HIVE-7591
>             Project: Hive
>          Issue Type: Bug
>          Components: Spark
>            Reporter: Chao
>         Attachments: HIVE-7591-spark.patch
>
>
> When running queries, I got exception like this:
> FAILED: ClassCastException org.apache.hadoop.hive.ql.plan.SparkWork cannot be cast to
org.apache.hadoop.hive.ql.plan.TezWork
> 14/07/31 15:08:53 ERROR ql.Driver: FAILED: ClassCastException org.apache.hadoop.hive.ql.plan.SparkWork
cannot be cast to org.apache.hadoop.hive.ql.plan.TezWork
> java.lang.ClassCastException: org.apache.hadoop.hive.ql.plan.SparkWork cannot be cast
to org.apache.hadoop.hive.ql.plan.TezWork
> 	at org.apache.hadoop.hive.ql.optimizer.GenMapRedUtils.addStatsTask(GenMapRedUtils.java:1419)
> 	at org.apache.hadoop.hive.ql.optimizer.GenMapRedUtils.isMergeRequired(GenMapRedUtils.java:1645)
> 	at org.apache.hadoop.hive.ql.parse.spark.GenSparkUtils.processFileSink(GenSparkUtils.java:313)
> 	at org.apache.hadoop.hive.ql.parse.spark.SparkCompiler.generateTaskTree(SparkCompiler.java:180)
> 	at org.apache.hadoop.hive.ql.parse.TaskCompiler.compile(TaskCompiler.java:199)
> 	at org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.analyzeInternal(SemanticAnalyzer.java:9514)
> 	at org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:207)
> 	at org.apache.hadoop.hive.ql.parse.ExplainSemanticAnalyzer.analyzeInternal(ExplainSemanticAnalyzer.java:74)
> 	at org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:207)
> 	at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:413)
> 	at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:309)
> 	at org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:984)
> 	at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1049)
> 	at org.apache.hadoop.hive.ql.Driver.run(Driver.java:921)
> 	at org.apache.hadoop.hive.ql.Driver.run(Driver.java:911)
> 	at org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:265)
> 	at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:217)
> 	at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:427)
> 	at org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:800)
> 	at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:694)
> 	at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:633)
> 	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> 	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> 	at java.lang.reflect.Method.invoke(Method.java:606)
> 	at org.apache.hadoop.util.RunJar.main(RunJar.java:212)
> Apparently, GenMapRedUtils::addStatsTask only assumes either MapredWork or TezWork, and
since we are introducing SparkWork, this need to be fixed.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

Mime
View raw message