hive-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "kangyanli (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (HIVE-12747) Hive on Spark, No plan file found ...map.xml
Date Fri, 25 Dec 2015 08:48:49 GMT

     [ https://issues.apache.org/jira/browse/HIVE-12747?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

kangyanli updated HIVE-12747:
-----------------------------
    Description: 
I use hive-1.2.1 and spark-1.3.1, hadoop-2.6.2. got the error below on worker. I noticed that
the map.xml file was here once, but was deleted later. so get the error.
15/12/25 16:35:14 INFO exec.Utilities: No plan file found: hdfs://hadoop11:32007/tmp/hive/kangyanli/bb8a2175-7d48-4a5a-96e7-99a965bb330f/hive_2015-12-25_16-33-13_585_4716428893980638808-1/-mr-10003/5e23b623-1dc1-428d-aef7-b8b73e43b3dd/map.xml
15/12/25 16:35:14 ERROR executor.Executor: Exception in task 2.0 in stage 0.0 (TID 2)
java.lang.NullPointerException
        at org.apache.hadoop.hive.ql.io.HiveInputFormat.init(HiveInputFormat.java:255)
        at org.apache.hadoop.hive.ql.io.HiveInputFormat.pushProjectionsAndFilters(HiveInputFormat.java:437)
        at org.apache.hadoop.hive.ql.io.HiveInputFormat.pushProjectionsAndFilters(HiveInputFormat.java:430)
        at org.apache.hadoop.hive.ql.io.CombineHiveInputFormat.getRecordReader(CombineHiveInputFormat.java:587)
        at org.apache.spark.rdd.HadoopRDD$$anon$1.<init>(HadoopRDD.scala:236)
        at org.apache.spark.rdd.HadoopRDD.compute(HadoopRDD.scala:212)
        at org.apache.spark.rdd.HadoopRDD.compute(HadoopRDD.scala:101)
        at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:277)
        at org.apache.spark.rdd.RDD.iterator(RDD.scala:244)
        at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:35)
        at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:277)
        at org.apache.spark.rdd.RDD.iterator(RDD.scala:244)
        at org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:68)
        at org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:41)
        at org.apache.spark.scheduler.Task.run(Task.scala:64)
        at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:203)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
        at java.lang.Thread.run(Thread.java:744)


  was:
I use hive-1.2.1 and spark-1.3.1, hadoop-2.6.2. got the error below on worker. I noticed that
the map.xml file was here once, but was deleted.
15/12/25 16:35:14 INFO exec.Utilities: No plan file found: hdfs://hadoop11:32007/tmp/hive/kangyanli/bb8a2175-7d48-4a5a-96e7-99a965bb330f/hive_2015-12-25_16-33-13_585_4716428893980638808-1/-mr-10003/5e23b623-1dc1-428d-aef7-b8b73e43b3dd/map.xml
15/12/25 16:35:14 ERROR executor.Executor: Exception in task 2.0 in stage 0.0 (TID 2)
java.lang.NullPointerException
        at org.apache.hadoop.hive.ql.io.HiveInputFormat.init(HiveInputFormat.java:255)
        at org.apache.hadoop.hive.ql.io.HiveInputFormat.pushProjectionsAndFilters(HiveInputFormat.java:437)
        at org.apache.hadoop.hive.ql.io.HiveInputFormat.pushProjectionsAndFilters(HiveInputFormat.java:430)
        at org.apache.hadoop.hive.ql.io.CombineHiveInputFormat.getRecordReader(CombineHiveInputFormat.java:587)
        at org.apache.spark.rdd.HadoopRDD$$anon$1.<init>(HadoopRDD.scala:236)
        at org.apache.spark.rdd.HadoopRDD.compute(HadoopRDD.scala:212)
        at org.apache.spark.rdd.HadoopRDD.compute(HadoopRDD.scala:101)
        at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:277)
        at org.apache.spark.rdd.RDD.iterator(RDD.scala:244)
        at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:35)
        at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:277)
        at org.apache.spark.rdd.RDD.iterator(RDD.scala:244)
        at org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:68)
        at org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:41)
        at org.apache.spark.scheduler.Task.run(Task.scala:64)
        at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:203)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
        at java.lang.Thread.run(Thread.java:744)



> Hive on Spark, No plan file found ...map.xml
> --------------------------------------------
>
>                 Key: HIVE-12747
>                 URL: https://issues.apache.org/jira/browse/HIVE-12747
>             Project: Hive
>          Issue Type: Bug
>            Reporter: kangyanli
>
> I use hive-1.2.1 and spark-1.3.1, hadoop-2.6.2. got the error below on worker. I noticed
that the map.xml file was here once, but was deleted later. so get the error.
> 15/12/25 16:35:14 INFO exec.Utilities: No plan file found: hdfs://hadoop11:32007/tmp/hive/kangyanli/bb8a2175-7d48-4a5a-96e7-99a965bb330f/hive_2015-12-25_16-33-13_585_4716428893980638808-1/-mr-10003/5e23b623-1dc1-428d-aef7-b8b73e43b3dd/map.xml
> 15/12/25 16:35:14 ERROR executor.Executor: Exception in task 2.0 in stage 0.0 (TID 2)
> java.lang.NullPointerException
>         at org.apache.hadoop.hive.ql.io.HiveInputFormat.init(HiveInputFormat.java:255)
>         at org.apache.hadoop.hive.ql.io.HiveInputFormat.pushProjectionsAndFilters(HiveInputFormat.java:437)
>         at org.apache.hadoop.hive.ql.io.HiveInputFormat.pushProjectionsAndFilters(HiveInputFormat.java:430)
>         at org.apache.hadoop.hive.ql.io.CombineHiveInputFormat.getRecordReader(CombineHiveInputFormat.java:587)
>         at org.apache.spark.rdd.HadoopRDD$$anon$1.<init>(HadoopRDD.scala:236)
>         at org.apache.spark.rdd.HadoopRDD.compute(HadoopRDD.scala:212)
>         at org.apache.spark.rdd.HadoopRDD.compute(HadoopRDD.scala:101)
>         at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:277)
>         at org.apache.spark.rdd.RDD.iterator(RDD.scala:244)
>         at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:35)
>         at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:277)
>         at org.apache.spark.rdd.RDD.iterator(RDD.scala:244)
>         at org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:68)
>         at org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:41)
>         at org.apache.spark.scheduler.Task.run(Task.scala:64)
>         at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:203)
>         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
>         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
>         at java.lang.Thread.run(Thread.java:744)



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message