hive-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Mich Talebzadeh <mich.talebza...@gmail.com>
Subject Re: Anyone successfully deployed Hive on TEZ engine?
Date Mon, 30 May 2016 09:38:04 GMT
Thanks Gopal.

I do not use any vendor's product., All my own set up, build and configure.
No CDH no HDL etc.

This the current stack that I have:

Java

*java -version*
java version "1.8.0_77"
Java(TM) SE Runtime Environment (build 1.8.0_77-b03)
Java HotSpot(TM) 64-Bit Server VM (build 25.77-b03, mixed mode)

HDFS version


*hadoop version*Hadoop 2.6.0
Subversion https://git-wip-us.apache.org/repos/asf/hadoop.git -r
e3496499ecb8d220fba99dc5ed4c99c8f9e33bb1
Compiled by jenkins on 2014-11-13T21:10Z
Compiled with protoc 2.5.0

YARN version


*yarn version*Hadoop 2.6.0
Subversion https://git-wip-us.apache.org/repos/asf/hadoop.git -r
e3496499ecb8d220fba99dc5ed4c99c8f9e33bb1
Compiled by jenkins on 2014-11-13T21:10Z
Compiled with protoc 2.5.0

HIVE version

hive --version
Hive 2.0.0
Subversion git://reznor-mbp-2.local/Users/sergey/git/hivegit -r
7f9f1fcb8697fb33f0edc2c391930a3728d247d7
Compiled by sergey on Tue Feb 9 18:12:08 PST 2016


Spark version

version 1.6.1
Using Scala version 2.10.5 (Java HotSpot(TM) 64-Bit Server VM, Java
1.8.0_77)


TEZ I downloaded

tez-0.8.3

And built it using the following instructions

https://tez.apache.org/install.html


Ok I just need to make it work as I have hive on spark engine as well.

please tell me what version of tez and yarn etc. I

thanks



Dr Mich Talebzadeh



LinkedIn * https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw
<https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw>*



http://talebzadehmich.wordpress.com



On 30 May 2016 at 10:16, Gopal Vijayaraghavan <gopalv@apache.org> wrote:

>
> > In short at the simplest set up what Resource Manager it works with?
>
> Tez+Hive needs HDFS and YARN 2.6.0+ (preferably as close to an Apache
> build as possible - CDH clusters need more work).
>
> Hive2 needs Apache Slider 0.91 right now, to start the cache daemons on
> YARN (see SLIDER-82).
>
> > If so kindly specify both Hive and TEZ versions.
>
> I maintain build scripts & configuration setups for Hive+Tez, for work
>
> https://github.com/t3rmin4t0r/tez-autobuild/tree/llap
>
>
> Both that & the master there builds Hive (2.1.0-SNAPSHOT) + Tez
> (0.8.4-SNAPSHOT), this one has the
> LLAP cache configurations turned on.
>
> This is what I use to develop Hive, before there are releases and it will
> allow each user
> on a shared cluster to maintain their own independent private install of
> hive - if you look at
> something like the old Spotify Hive query presentations, you'll see that
> more people have
> used that to run their own private builds successfully :)
>
> Purely out of laziness, the LLAP configurations in slider-gen.sh (i.e the
> Xmx & cache values)
> are configured exactly to match my dev cluster - 32 vcore + 256Gb RAM.
>
> Cheers,
> Gopal
>
>
>

Mime
View raw message