Return-Path: X-Original-To: apmail-hive-user-archive@www.apache.org Delivered-To: apmail-hive-user-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 4949A10099 for ; Thu, 4 Dec 2014 01:52:40 +0000 (UTC) Received: (qmail 37697 invoked by uid 500); 4 Dec 2014 01:52:38 -0000 Delivered-To: apmail-hive-user-archive@hive.apache.org Received: (qmail 37636 invoked by uid 500); 4 Dec 2014 01:52:38 -0000 Mailing-List: contact user-help@hive.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@hive.apache.org Delivered-To: mailing list user@hive.apache.org Received: (qmail 37626 invoked by uid 99); 4 Dec 2014 01:52:38 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 04 Dec 2014 01:52:38 +0000 X-ASF-Spam-Status: No, hits=1.5 required=5.0 tests=HTML_MESSAGE,RCVD_IN_DNSWL_LOW,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (athena.apache.org: domain of yuemeng1@huawei.com designates 119.145.14.66 as permitted sender) Received: from [119.145.14.66] (HELO szxga03-in.huawei.com) (119.145.14.66) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 04 Dec 2014 01:52:31 +0000 Received: from 172.24.2.119 (EHLO SZXEML455-HUB.china.huawei.com) ([172.24.2.119]) by szxrg03-dlp.huawei.com (MOS 4.4.3-GA FastPath queued) with ESMTP id AYD37264; Thu, 04 Dec 2014 09:42:40 +0800 (CST) Received: from [127.0.0.1] (10.177.39.227) by SZXEML455-HUB.china.huawei.com (10.82.67.198) with Microsoft SMTP Server id 14.3.158.1; Thu, 4 Dec 2014 09:42:36 +0800 Message-ID: <547FBC09.1000609@huawei.com> Date: Thu, 4 Dec 2014 09:42:33 +0800 From: yuemeng1 User-Agent: Mozilla/5.0 (Windows NT 6.1; rv:17.0) Gecko/20130509 Thunderbird/17.0.6 MIME-Version: 1.0 To: Subject: Re: Job aborted due to stage failure References: <547D2263.9090104@huawei.com> <547D34DC.8070509@huawei.com> <547DACFC.6080909@huawei.com> <547E75A5.4000505@huawei.com> In-Reply-To: Content-Type: multipart/alternative; boundary="------------010807040804050106070509" X-Originating-IP: [10.177.39.227] X-CFilter-Loop: Reflected X-Mirapoint-Virus-RAPID-Raw: score=unknown(0), refid=str=0001.0A020209.547FBC11.004B,ss=1,re=0.000,recu=0.000,reip=0.000,cl=1,cld=1,fgs=0, ip=0.0.0.0, so=2013-05-26 15:14:31, dmn=2013-03-21 17:37:32 X-Mirapoint-Loop-Id: ebb5a0c38b06d2aa8d0da6407d3cb719 X-Virus-Checked: Checked by ClamAV on apache.org --------------010807040804050106070509 Content-Type: text/plain; charset="UTF-8"; format=flowed Content-Transfer-Encoding: 7bit hi,thanks a lot for your help,with your help ,my hive-on-spark can work well now it take me long time to install and deploy.here are some advice,i think we need to improve the installation documentation, allowing users to use the least amount of time to compile and install 1)add which spark version we should pick from spark github if we select built spark instead of download a spark pre-built,tell them the right built commad!(not include Pyarn ,Phive) 2)if they get some error during built ,such as [ERRO/hive/ql/src/java/org/apache/hadoop/hive/ql/exec/spark/status/SparkJobStatus.java:[22,24]cannot find symbol [ERROR]symbol: class JobExecutionStatus,tell them what they can do? for our users,first to use it ,then feel good or bad? and if u need,i can add something to start document thanks yuemeng On 2014/12/3 11:03, Xuefu Zhang wrote: > When you build Spark, remove -Phive as well as -Pyarn. When you run > hive queries, you may need to run "set spark.home=/path/to/spark/dir"; > > Thanks, > Xuefu > > On Tue, Dec 2, 2014 at 6:29 PM, yuemeng1 > wrote: > > hi,XueFu,thanks a lot for your help,now i will provide more detail > to reproduce this ssue: > 1),i checkout a spark branch from hive > github(https://github.com/apache/hive/tree/spark on Nov 29,becasue > of for version now it will give something wrong about:Caused by: > java.lang.RuntimeException: Unable to instantiate > org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient ), > and built command:mvn clean package -DskipTests -Phadoop-2 -Pdist > after built i get package from > :/home/ym/hive-on-spark/hive1129/hive/packaging/target(apache-hive-0.15.0-SNAPSHOT-bin.tar.gz) > 2)i checkout spark from > https://github.com/apache/spark/tree/v1.2.0-snapshot0,becasue of > spark branch-1.2 is with spark parent(1.2.1-SNAPSHOT),so i chose > v1.2.0-snapshot0 and i compare this spark's pom.xml with > spark-parent-1.2.0-SNAPSHOT.pom(get from > http://ec2-50-18-79-139.us-west-1.compute.amazonaws.com/data/spark_2.10-1.2-SNAPSHOT/org/apache/spark/spark-parent/1.2.0-SNAPSHOT/),and > there is only difference is spark-parent name,and built command is : > > |mvn -Pyarn -Phadoop-2.4 -Dhadoop.version=2.4.0 -Phive -DskipTests clean package| > > 3)comand i execute in hive-shell: > ./hive --auxpath > /opt/hispark/spark/assembly/target/scala-2.10/spark-assembly-1.2.0-hadoop2.4.0.jar(copy > this jar to hive dir lib already) > create table student(sno int,sname string,sage int,ssex string) > row format delimited FIELDS TERMINATED BY ','; > create table score(sno int,cno int,sage int) row format delimited > FIELDS TERMINATED BY ','; > load data local inpath > '/home/hive-on-spark/temp/spark-1.2.0/examples/src/main/resources/student.txt' > into table student; > load data local inpath > '/home/hive-on-spark/temp/spark-1.2.0/examples/src/main/resources/score.txt' > into table score; > set hive.execution.engine=spark; > set spark.master=spark://10.175.xxx.xxx:7077; > set spark.eventLog.enabled=true; > set spark.executor.memory=9086m; > set spark.serializer=org.apache.spark.serializer.KryoSerializer; > select distinct st.sno,sname from student st join score sc > on(st.sno=sc.sno) where sc.cno IN(11,12,13) and st.sage > 28;(work > in mr) > 4) > studdent.txt file > 1,rsh,27,female > 2,kupo,28,male > 3,astin,29,female > 4,beike,30,male > 5,aili,31,famle > > score.txt file > 1,10,80 > 2,11,85 > 3,12,90 > 4,13,95 > 5,14,100 > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On 2014/12/2 23:28, Xuefu Zhang wrote: >> Could you provide details on how to reproduce the issue? such as >> the exact spark branch, the command to build Spark, how you build >> Hive, and what queries/commands you run. >> >> We are running Hive on Spark all the time. Our pre-commit test >> runs without any issue. >> >> Thanks, >> Xuefu >> >> On Tue, Dec 2, 2014 at 4:13 AM, yuemeng1 > > wrote: >> >> hi,XueFu >> i checkout a spark branch from >> sparkgithub(tags:v1.2.0-snapshot0)and i compare this spark's >> pom.xml with spark-parent-1.2.0-SNAPSHOT.pom(get from >> http://ec2-50-18-79-139.us-west-1.compute.amazonaws.com/data/spark_2.10-1.2-SNAPSHOT/org/apache/spark/spark-parent/1.2.0-SNAPSHOT/),and >> there is only difference is follow: >> in spark-parent-1.2.0-SNAPSHOT.pom >> spark-parent >> 1.2.0-SNAPSHOT >> and in v1.2.0-snapshot0 >> spark-parent >> 1.2.0 >> i think there is no essence diff,and i built v1.2.0-snapshot0 >> and deploy it as my spark clusters >> when i run query about join two table ,it still give some >> error what i show u earlier >> >> Job aborted due to stage failure: Task 0 in stage 1.0 failed >> 4 times, most recent failure: Lost task 0.3 in stage 1.0 (TID >> 7, datasight18): java.lang.NullPointerException+details >> >> Job aborted due to stage failure: Task 0 in stage 1.0 failed 4 times, most recent failure: Lost task 0.3 in stage 1.0 (TID 7, datasight18): java.lang.NullPointerException >> at org.apache.hadoop.hive.ql.io.HiveInputFormat.init(HiveInputFormat.java:255) >> at org.apache.hadoop.hive.ql.io.HiveInputFormat.pushProjectionsAndFilters(HiveInputFormat.java:437) >> at org.apache.hadoop.hive.ql.io.HiveInputFormat.pushProjectionsAndFilters(HiveInputFormat.java:430) >> at org.apache.hadoop.hive.ql.io.CombineHiveInputFormat.getRecordReader(CombineHiveInputFormat.java:587) >> at org.apache.spark.rdd.HadoopRDD$$anon$1.(HadoopRDD.scala:233) >> at org.apache.spark.rdd.HadoopRDD.compute(HadoopRDD.scala:210) >> at org.apache.spark.rdd.HadoopRDD.compute(HadoopRDD.scala:99) >> at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:263) >> at org.apache.spark.rdd.RDD.iterator(RDD.scala:230) >> at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:35) >> at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:263) >> at org.apache.spark.rdd.RDD.iterator(RDD.scala:230) >> at org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:68) >> at org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:41) >> at org.apache.spark.scheduler.Task.run(Task.scala:56) >> at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:196) >> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110) >> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603) >> at java.lang.Thread.run(Thread.java:722) >> >> Driver stacktrace: >> >> >> >> i think my spark clusters did't had any problem,but why >> always give me such error >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> On 2014/12/2 13:39, Xuefu Zhang wrote: >>> You need to build your spark assembly from spark 1.2 branch. >>> this should give your both a spark build as well as >>> spark-assembly jar, which you need to copy to Hive lib >>> directory. Snapshot is fine, and spark 1.2 hasn't been >>> released yet. >>> >>> --Xuefu >>> >>> On Mon, Dec 1, 2014 at 7:41 PM, yuemeng1 >>> > wrote: >>> >>> >>> >>> hi.XueFu, >>> thanks a lot for your inforamtion,but as far as i know >>> ,the latest spark version on github is >>> spark-snapshot-1.3,but there is no spark-1.2,only have a >>> branch-1.2 with spark-snapshot-1.2,can u tell me which >>> spark version i should built,and for now,that's >>> spark-assembly-1.2.0-SNAPSHOT-hadoop2.4.0.jar produce >>> error like that >>> >>> >>> On 2014/12/2 11:03, Xuefu Zhang wrote: >>>> It seems that wrong class, HiveInputFormat, is loaded. >>>> The stacktrace is way off the current Hive code. You >>>> need to build Spark 1.2 and copy spark-assembly jar to >>>> Hive's lib directory and that it. >>>> >>>> --Xuefu >>>> >>>> On Mon, Dec 1, 2014 at 6:22 PM, yuemeng1 >>>> > wrote: >>>> >>>> hi,i built a hive on spark package and my spark >>>> assembly jar is >>>> spark-assembly-1.2.0-SNAPSHOT-hadoop2.4.0.jar,when >>>> i run a query in hive shell,before execute this query, >>>> i set all the require which hive need with >>>> spark.and i execute a join query : >>>> select distinct st.sno,sname from student st join >>>> score sc on(st.sno=sc.sno) where sc.cno >>>> IN(11,12,13) and st.sage > 28; >>>> but it failed, >>>> get follow error in spark webUI: >>>> Job aborted due to stage failure: Task 0 in stage >>>> 1.0 failed 4 times, most recent failure: Lost task >>>> 0.3 in stage 1.0 (TID 7, datasight18): >>>> java.lang.NullPointerException+details >>>> >>>> Job aborted due to stage failure: Task 0 in stage 1.0 failed 4 times, most recent failure: Lost task 0.3 in stage 1.0 (TID 7, datasight18): java.lang.NullPointerException >>>> at org.apache.hadoop.hive.ql.io.HiveInputFormat.init(HiveInputFormat.java:255) >>>> at org.apache.hadoop.hive.ql.io.HiveInputFormat.pushProjectionsAndFilters(HiveInputFormat.java:437) >>>> at org.apache.hadoop.hive.ql.io.HiveInputFormat.pushProjectionsAndFilters(HiveInputFormat.java:430) >>>> at org.apache.hadoop.hive.ql.io.CombineHiveInputFormat.getRecordReader(CombineHiveInputFormat.java:587) >>>> at org.apache.spark.rdd.HadoopRDD$$anon$1.(HadoopRDD.scala:233) >>>> at org.apache.spark.rdd.HadoopRDD.compute(HadoopRDD.scala:210) >>>> at org.apache.spark.rdd.HadoopRDD.compute(HadoopRDD.scala:99) >>>> at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:263) >>>> at org.apache.spark.rdd.RDD.iterator(RDD.scala:230) >>>> at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:35) >>>> at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:263) >>>> at org.apache.spark.rdd.RDD.iterator(RDD.scala:230) >>>> at org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:68) >>>> at org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:41) >>>> at org.apache.spark.scheduler.Task.run(Task.scala:56) >>>> at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:196) >>>> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110) >>>> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603) >>>> at java.lang.Thread.run(Thread.java:722) >>>> >>>> Driver stacktrace: >>>> >>>> can u give me a help to deal this probelm,and i >>>> think my built was succussed! >>>> >>>> >>> >>> >> >> > > --------------010807040804050106070509 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: 8bit
hi,thanks a lot for your help,with your help ,my hive-on-spark can work well now
it take me long time to install and deploy.here are  some advice,i think we need to improve the installation documentation, allowing users to use the least amount of time to compile and install
1)add which spark version we should pick from spark github if we select built spark instead of download a spark pre-built,tell them the right built commad!(not include Pyarn ,Phive)
2)if they get some error during built ,such as [ERRO/hive/ql/src/java/org/apache/hadoop/hive/ql/exec/spark/status/SparkJobStatus.java:[22,24] cannot find symbol
[ERROR] symbol: class JobExecutionStatus,tell them what they can do?
for our users,first to use it ,then  feel good or bad?
and if u need,i can add something to start document


thanks
yuemeng





On 2014/12/3 11:03, Xuefu Zhang wrote:
When you build Spark, remove -Phive as well as -Pyarn. When you run hive queries, you may need to run "set spark.home=/path/to/spark/dir";

Thanks,
Xuefu

On Tue, Dec 2, 2014 at 6:29 PM, yuemeng1 <yuemeng1@huawei.com> wrote:
hi,XueFu,thanks a lot for your help,now i will provide more detail to reproduce this ssue:
1),i checkout a spark branch from hive github(https://github.com/apache/hive/tree/spark on Nov 29,becasue of for version now it will give something wrong about:Caused by: java.lang.RuntimeException: Unable to instantiate org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient ),
and built command:mvn clean package -DskipTests -Phadoop-2 -Pdist
after built i get package from :/home/ym/hive-on-spark/hive1129/hive/packaging/target(apache-hive-0.15.0-SNAPSHOT-bin.tar.gz)
2)i checkout spark from https://github.com/apache/spark/tree/v1.2.0-snapshot0,becasue of spark branch-1.2 is with spark parent(1.2.1-SNAPSHOT),so i chose v1.2.0-snapshot0 and i compare this spark's pom.xml with spark-parent-1.2.0-SNAPSHOT.pom(get from http://ec2-50-18-79-139.us-west-1.compute.amazonaws.com/data/spark_2.10-1.2-SNAPSHOT/org/apache/spark/spark-parent/1.2.0-SNAPSHOT/),and there is only difference is spark-parent name,and built command is :
mvn -Pyarn -Phadoop-2.4 -Dhadoop.version=2.4.0 -Phive -DskipTests clean package
3)comand i execute in hive-shell:
./hive --auxpath /opt/hispark/spark/assembly/target/scala-2.10/spark-assembly-1.2.0-hadoop2.4.0.jar(copy this jar to hive dir lib already)
create table student(sno int,sname string,sage int,ssex string) row format delimited FIELDS TERMINATED BY ',';
create table score(sno int,cno int,sage int) row format delimited FIELDS TERMINATED BY ',';
load data local inpath '/home/hive-on-spark/temp/spark-1.2.0/examples/src/main/resources/student.txt' into table student;
load data local inpath '/home/hive-on-spark/temp/spark-1.2.0/examples/src/main/resources/score.txt' into table score;
set hive.execution.engine=spark;
set spark.master=spark://10.175.xxx.xxx:7077;
set spark.eventLog.enabled=true;
set spark.executor.memory=9086m;
set spark.serializer=org.apache.spark.serializer.KryoSerializer;
select distinct st.sno,sname from student st join score sc on(st.sno=sc.sno) where sc.cno IN(11,12,13) and st.sage > 28;(work in mr)
4)
studdent.txt file
1,rsh,27,female
2,kupo,28,male
3,astin,29,female
4,beike,30,male
5,aili,31,famle

score.txt file
1,10,80
2,11,85
3,12,90
4,13,95
5,14,100






























On 2014/12/2 23:28, Xuefu Zhang wrote:
Could you provide details on how to reproduce the issue? such as the exact spark branch, the command to build Spark, how you build Hive, and what queries/commands you run.

We are running Hive on Spark all the time. Our pre-commit test runs without any issue.

Thanks,
Xuefu

On Tue, Dec 2, 2014 at 4:13 AM, yuemeng1 <yuemeng1@huawei.com> wrote:
hi,XueFu
i checkout a spark branch from sparkgithub(tags:v1.2.0-snapshot0)and i compare this spark's pom.xml with spark-parent-1.2.0-SNAPSHOT.pom(get from http://ec2-50-18-79-139.us-west-1.compute.amazonaws.com/data/spark_2.10-1.2-SNAPSHOT/org/apache/spark/spark-parent/1.2.0-SNAPSHOT/),and there is only difference is follow:
in spark-parent-1.2.0-SNAPSHOT.pom
  <artifactId>spark-parent</artifactId>
  <version>1.2.0-SNAPSHOT</version>
and in v1.2.0-snapshot0
<artifactId>spark-parent</artifactId>
  <version>1.2.0</version>
i think there is no essence diff,and i built v1.2.0-snapshot0 and deploy it as my spark clusters
when i run query about join two table ,it still give some error what i show u earlier

Job aborted due to stage failure: Task 0 in stage 1.0 failed 4 times, most recent failure: Lost task 0.3 in stage 1.0 (TID 7, datasight18): java.lang.NullPointerException+details 
Job aborted due to stage failure: Task 0 in stage 1.0 failed 4 times, most recent failure: Lost task 0.3 in stage 1.0 (TID 7, datasight18): java.lang.NullPointerException
	at org.apache.hadoop.hive.ql.io.HiveInputFormat.init(HiveInputFormat.java:255)
	at org.apache.hadoop.hive.ql.io.HiveInputFormat.pushProjectionsAndFilters(HiveInputFormat.java:437)
	at org.apache.hadoop.hive.ql.io.HiveInputFormat.pushProjectionsAndFilters(HiveInputFormat.java:430)
	at org.apache.hadoop.hive.ql.io.CombineHiveInputFormat.getRecordReader(CombineHiveInputFormat.java:587)
	at org.apache.spark.rdd.HadoopRDD$$anon$1.<init>(HadoopRDD.scala:233)
	at org.apache.spark.rdd.HadoopRDD.compute(HadoopRDD.scala:210)
	at org.apache.spark.rdd.HadoopRDD.compute(HadoopRDD.scala:99)
	at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:263)
	at org.apache.spark.rdd.RDD.iterator(RDD.scala:230)
	at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:35)
	at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:263)
	at org.apache.spark.rdd.RDD.iterator(RDD.scala:230)
	at org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:68)
	at org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:41)
	at org.apache.spark.scheduler.Task.run(Task.scala:56)
	at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:196)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
	at java.lang.Thread.run(Thread.java:722)

Driver stacktrace:


 i think my spark clusters did't had any problem,but why always give me such error






















On 2014/12/2 13:39, Xuefu Zhang wrote:
You need to build your spark assembly from spark 1.2 branch. this should give your both a spark build as well as spark-assembly jar, which you need to copy to Hive lib directory. Snapshot is fine, and spark 1.2 hasn't been released yet.

--Xuefu

On Mon, Dec 1, 2014 at 7:41 PM, yuemeng1 <yuemeng1@huawei.com> wrote:


hi.XueFu,
thanks a lot for your inforamtion,but as far as i know ,the latest spark version on github is spark-snapshot-1.3,but there is no spark-1.2,only have a branch-1.2 with spark-snapshot-1.2,can u tell me which spark version i should built,and for now,that's spark-assembly-1.2.0-SNAPSHOT-hadoop2.4.0.jar produce error like that


On 2014/12/2 11:03, Xuefu Zhang wrote:
It seems that wrong class, HiveInputFormat, is loaded. The stacktrace is way off the current Hive code. You need to build Spark 1.2 and copy spark-assembly jar to Hive's lib directory and that it.

--Xuefu

On Mon, Dec 1, 2014 at 6:22 PM, yuemeng1 <yuemeng1@huawei.com> wrote:
hi,i built a hive on spark package and my spark assembly jar is spark-assembly-1.2.0-SNAPSHOT-hadoop2.4.0.jar,when i run a query in hive shell,before execute this query,
i set all the  require which hive need with  spark.and i execute a join query :
select distinct st.sno,sname from student st join score sc on(st.sno=sc.sno) where sc.cno IN(11,12,13) and st.sage > 28;
but it failed,
get follow error in spark webUI:
Job aborted due to stage failure: Task 0 in stage 1.0 failed 4 times, most recent failure: Lost task 0.3 in stage 1.0 (TID 7, datasight18): java.lang.NullPointerException+details 
Job aborted due to stage failure: Task 0 in stage 1.0 failed 4 times, most recent failure: Lost task 0.3 in stage 1.0 (TID 7, datasight18): java.lang.NullPointerException
	at org.apache.hadoop.hive.ql.io.HiveInputFormat.init(HiveInputFormat.java:255)
	at org.apache.hadoop.hive.ql.io.HiveInputFormat.pushProjectionsAndFilters(HiveInputFormat.java:437)
	at org.apache.hadoop.hive.ql.io.HiveInputFormat.pushProjectionsAndFilters(HiveInputFormat.java:430)
	at org.apache.hadoop.hive.ql.io.CombineHiveInputFormat.getRecordReader(CombineHiveInputFormat.java:587)
	at org.apache.spark.rdd.HadoopRDD$$anon$1.<init>(HadoopRDD.scala:233)
	at org.apache.spark.rdd.HadoopRDD.compute(HadoopRDD.scala:210)
	at org.apache.spark.rdd.HadoopRDD.compute(HadoopRDD.scala:99)
	at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:263)
	at org.apache.spark.rdd.RDD.iterator(RDD.scala:230)
	at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:35)
	at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:263)
	at org.apache.spark.rdd.RDD.iterator(RDD.scala:230)
	at org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:68)
	at org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:41)
	at org.apache.spark.scheduler.Task.run(Task.scala:56)
	at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:196)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
	at java.lang.Thread.run(Thread.java:722)

Driver stacktrace:
can u give me a help to deal this probelm,and i think my built was succussed!








--------------010807040804050106070509--