hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Rui Li (JIRA)" <>
Subject [jira] [Created] (HIVE-7773) Union all query finished with errors [Spark Branch]
Date Tue, 19 Aug 2014 02:08:18 GMT
Rui Li created HIVE-7773:

             Summary: Union all query finished with errors [Spark Branch]
                 Key: HIVE-7773
             Project: Hive
          Issue Type: Bug
          Components: Spark
            Reporter: Rui Li
            Priority: Critical

When I run a union all query, I found the following error in spark log (the query finished
with correct results though):
java.lang.RuntimeException: Map operator initialization failed
        at org.apache.hadoop.hive.ql.exec.spark.SparkMapRecordHandler.init(
        at org.apache.spark.rdd.RDD$$anonfun$13.apply(RDD.scala:596)
        at org.apache.spark.rdd.RDD$$anonfun$13.apply(RDD.scala:596)
        at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:35)
        at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:262)
        at org.apache.spark.rdd.RDD.iterator(RDD.scala:229)
        at org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:68)
        at org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:41)
        at org.apache.spark.executor.Executor$
        at java.util.concurrent.ThreadPoolExecutor.runWorker(
        at java.util.concurrent.ThreadPoolExecutor$
Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: org.apache.hadoop.hive.ql.metadata.HiveException:
Configuration and input path are inconsistent
        at org.apache.hadoop.hive.ql.exec.MapOperator.setChildren(
        at org.apache.hadoop.hive.ql.exec.spark.SparkMapRecordHandler.init(
        ... 16 more
Judging from the log, I think we don't properly handle the input paths when cloning the job
conf, so it may also affect other queries with multiple maps or reduces.

This message was sent by Atlassian JIRA

View raw message