hive-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Sofia Panagiotidi <sofia.panagiot...@taiger.com>
Subject Re: hive1.2.1 on spark 1.5.2
Date Tue, 26 Jan 2016 13:00:54 GMT

Using the prebuilt versions from the apache website for both Hive and Spark


> On 26 Jan 2016, at 14:00, Sofia Panagiotidi <sofia.panagiotidi@taiger.com> wrote:
> 
> I have also managed to use Hive 1.2.1 with Spark 1.4.1
> 
> 
>> On 26 Jan 2016, at 10:20, Mich Talebzadeh <mich@peridale.co.uk <mailto:mich@peridale.co.uk>>
wrote:
>> 
>> As far as I have worked this one out Hive 1.2.1 works on Spark 1.3.1 for now. This
means Hive will use spark engine.
>>  
>>  
>> set spark.home=/usr/lib/spark-1.3.1-bin-hadoop2.6;
>> set hive.execution.engine=spark;
>> set spark.master=yarn-client;
>> set hive.optimize.ppd=true;
>> Beeline version 1.2.1 by Apache Hive
>> select count(1) from smallsales;
>> INFO  :
>> Query Hive on Spark job[0] stages:
>> INFO  : 0
>> INFO  : 1
>> INFO  :
>> Status: Running (Hive on Spark job[0])
>> INFO  : Job Progress Format
>> CurrentTime StageId_StageAttemptId: SucceededTasksCount(+RunningTasksCount-FailedTasksCount)/TotalTasksCount
[StageCost]
>> INFO  : 2016-01-26 09:27:30,240 Stage-0_0: 0(+1)/1      Stage-1_0: 0/1
>> INFO  : 2016-01-26 09:27:33,258 Stage-0_0: 0(+1)/1      Stage-1_0: 0/1
>> INFO  : 2016-01-26 09:27:35,265 Stage-0_0: 1/1 Finished Stage-1_0: 0(+1)/1
>> INFO  : 2016-01-26 09:27:38,281 Stage-0_0: 1/1 Finished Stage-1_0: 1/1 Finished
>> INFO  : Status: Finished successfully in 28.17 seconds
>> +----------+--+
>> |   _c0    |
>> +----------+--+
>> | 5000000  |
>> +----------+--+
>>  
>> HTH
>>  
>> Dr Mich Talebzadeh
>>  
>> LinkedIn  https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw
<https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw>
>>  
>> Sybase ASE 15 Gold Medal Award 2008
>> A Winning Strategy: Running the most Critical Financial Data on ASE 15
>> http://login.sybase.com/files/Product_Overviews/ASE-Winning-Strategy-091908.pdf <http://login.sybase.com/files/Product_Overviews/ASE-Winning-Strategy-091908.pdf>
>> Author of the books "A Practitioner’s Guide to Upgrading to Sybase ASE 15", ISBN
978-0-9563693-0-7. 
>> co-author "Sybase Transact SQL Guidelines Best Practices", ISBN 978-0-9759693-0-4
>> Publications due shortly:
>> Complex Event Processing in Heterogeneous Environments, ISBN: 978-0-9563693-3-8
>> Oracle and Sybase, Concepts and Contrasts, ISBN: 978-0-9563693-1-4, volume one out
shortly
>>  
>> http://talebzadehmich.wordpress.com <http://talebzadehmich.wordpress.com/>
>>  
>> NOTE: The information in this email is proprietary and confidential. This message
is for the designated recipient only, if you are not the intended recipient, you should destroy
it immediately. Any information in this message shall not be understood as given or endorsed
by Peridale Technology Ltd, its subsidiaries or their employees, unless expressly so stated.
It is the responsibility of the recipient to ensure that this email is virus free, therefore
neither Peridale Technology Ltd, its subsidiaries nor their employees accept any responsibility.
>>  
>> From: kevin [mailto:kiss.kevin119@gmail.com <mailto:kiss.kevin119@gmail.com>]

>> Sent: 26 January 2016 08:45
>> To: user@spark.apache.org <mailto:user@spark.apache.org>; user@hive.apache.org
<mailto:user@hive.apache.org>
>> Subject: hive1.2.1 on spark 1.5.2
>>  
>> hi,all
>>    I tried hive on spark with version hive1.2.1 spark1.5.2. I build spark witout
-Phive . And I test spark cluster stand alone with spark-submit and it is ok.
>>    but when I use hive , on spark web-site I can see the hive on spark application
,finally I got error:
>>  
>> 16/01/26 16:23:42 INFO slf4j.Slf4jLogger: Slf4jLogger started
>> 16/01/26 16:23:42 INFO Remoting: Starting remoting
>> 16/01/26 16:23:42 INFO Remoting: Remoting started; listening on addresses :[akka.tcp://driverPropsFetcher@10.1.3.116:42307
<http://driverPropsFetcher@10.1.3.116:42307/>]
>> 16/01/26 16:23:42 INFO util.Utils: Successfully started service 'driverPropsFetcher'
on port 42307.
>> Exception in thread "main" akka.actor.ActorNotFound: Actor not found for: ActorSelection[Anchor(akka.tcp://sparkDriver@10.1.3.107:34725/
<http://sparkDriver@10.1.3.107:34725/>), Path(/user/CoarseGrainedScheduler)]
>>         at akka.actor.ActorSelection$$anonfun$resolveOne$1.apply(ActorSelection.scala:65)
>>         at akka.actor.ActorSelection$$anonfun$resolveOne$1.apply(ActorSelection.scala:63)
>>         at scala.concurrent.impl.CallbackRunnable.run(Promise.scala:32)
>>         at akka.dispatch.BatchingExecutor$AbstractBatch.processBatch(BatchingExecutor.scala:55)
>>         at akka.dispatch.BatchingExecutor$Batch.run(BatchingExecutor.scala:73)
>>         at akka.dispatch.ExecutionContexts$sameThreadExecutionContext$.unbatchedExecute(Future.scala:74)
>>         at akka.dispatch.BatchingExecutor$class.execute(BatchingExecutor.scala:120)
>>         at akka.dispatch.ExecutionContexts$sameThreadExecutionContext$.execute(Future.scala:73)
>>         at scala.concurrent.impl.CallbackRunnable.executeWithValue(Promise.scala:40)
>>         at scala.concurrent.impl.Promise$DefaultPromise.tryComplete(Promise.scala:248)
>>         at akka.pattern.PromiseActorRef.$bang(AskSupport.scala:266)
>>         at akka.remote.DefaultMessageDispatcher.dispatch(Endpoint.scala:89)
>>         at akka.remote.EndpointReader$$anonfun$receive$2.applyOrElse(Endpoint.scala:935)
>>         at akka.actor.Actor$class.aroundReceive(Actor.scala:467)
>>         at akka.remote.EndpointActor.aroundReceive(Endpoint.scala:411)
>>         at akka.actor.ActorCell.receiveMessage(ActorCell.scala:516)
>>         at akka.actor.ActorCell.invoke(ActorCell.scala:487)
>>         at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:238)
>>         at akka.dispatch.Mailbox.run(Mailbox.scala:220)
>>         at akka.dispatch.ForkJoinExecutorConfigurator$AkkaForkJoinTask.exec(AbstractDispatcher.scala:397)
>>         at scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)
>>         at scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339)
>>         at scala.concurrent.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)
>>         at scala.concurrent.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)
>>  
>>  
>> Could anyone tell me is it for the version of hive and spark not matching ? which
version is ok or there is some other reason? 
> 


Mime
View raw message