hive-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From hernan saab <hernan_javier_s...@yahoo.com>
Subject Re: Error with Hive 2.1.1 and Spark 2.1
Date Tue, 18 Apr 2017 19:44:31 GMT
Edward,Consider putting together a project at the time of documentation. This way, users at
least know all the variables that you use to prove a particular configuration works.Use GIT,
its a simple revision control tool that developers at the Hive community should get confortable
with. It should not take a great effort to snapshot the working version of your project and
link it in your documentation. 

    On Tuesday, April 18, 2017 12:41 PM, Edward Capriolo <edlinuxguru@gmail.com> wrote:
 

 

On Tue, Apr 18, 2017 at 3:32 PM, hernan saab <hernan_javier_saab@yahoo.com> wrote:

The effort of configuring an apache big data system by hand for your particular needs is equivalent
to herding rattlesnakes and cats into one small room.The documentation is poor and most of
the time the community developers don't really feel like helping you.Use Ambari or any other
orchestration tool you can find. It will save you a lot of angry moments and time.

 

    On Tuesday, April 18, 2017 11:45 AM, Vihang Karajgaonkar <vihang@cloudera.com> wrote:
 

 +sergio
Thank you for pointing this out. Based on what I see here https://github.com/ apache/hive/blob/branch-2.1/
pom.xml#L179 Hive 2.1 supports Sparks 1.6. There is a JIRA to add support for Spark 2.0 https://issues.apache.org/
jira/browse/HIVE-14029 but that is available from Hive 2.2.x
I have created https://issues.apache. org/ jira/browse/HIVE-16472 to fix the wiki for documentation
issues and any bugs in the code if needed.
On Mon, Apr 17, 2017 at 6:19 PM, hernan saab <hernan_javier_saab@yahoo.com> wrote:

IMO, that page is a booby trap for the newbies to make them waste their time needlessly.As
far as I know Hive on Spark does not work today.I would be the reason that page still stays
on is because there is a level of shame in the Hive dev community that a feature like this
should be functional by now.DO NOT USE SPARK ON HIVE.Instead use Tez on Hive.
Hernan
 

    On Monday, April 17, 2017 3:45 PM, Krishnanand Khambadkone <kkhambadkone@yahoo.com>
wrote:
 

 Hi,   I am trying to run Hive queries by using Spark as the execution engine.   I am following
the instructions on this page,
https://cwiki.apache.org/ confluence/display/Hive/Hive+ on+Spark%3A+Getting+Started

When I try to run my query which is. a simple count(*) command, I get this error,
Failed to execute spark task, with exception 'org.apache.hadoop.hive.ql. metadata.HiveException(Failed
to create spark client.)'FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.
exec.spark.SparkTask


   



   

When you choose a package or a tool you do not always get the version you want. We (hive pmc)
discussed "support" in our private list. In short, you can not expect software released 1
year ago to be drop-in-replaced by something released a month ago. 
The plan I am going to put forward is Hive binaries will come shipped with all of its dependencies.
Then one version of hive supports one version of X explicitly and for all other versions use
at your own risk.
Edward

   
Mime
View raw message