hive-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Artem Ervits <artemerv...@gmail.com>
Subject RE: Hive on TEZ fails starting
Date Tue, 05 Jan 2016 22:54:29 GMT
Check if you have conflicting java versions
On Jan 5, 2016 5:27 PM, "Mich Talebzadeh" <mich@peridale.co.uk> wrote:

> Hi Rajesh,
>
>
>
> This is what I have under :$HADOOP_COMMON_HOME/lib/native
>
>
>
> cd $HADOOP_COMMON_HOME/lib/native
>
> hduser@rhes564::/home/hduser/hadoop-2.6.0/lib/native> ls -ltr
>
> total 4936
>
> -rwxr-xr-x 1 hduser hadoop  278622 Nov 13  2014 libhdfs.so.0.0.0
>
> -rw-r-xr-x 1 hduser hadoop  440498 Nov 13  2014 libhdfs.a
>
> -rw-r-xr-x 1 hduser hadoop  476666 Nov 13  2014 libhadooputils.a
>
> -rw-r-xr-x 1 hduser hadoop 1634592 Nov 13  2014 libhadooppipes.a
>
> -rwxr-xr-x 1 hduser hadoop  805999 Nov 13  2014 libhadoop.so.1.0.0
>
> -rw-r-xr-x 1 hduser hadoop 1380212 Nov 13  2014 libhadoop.a
>
> lrwxrwxrwx 1 hduser hadoop      16 Feb  8  2015 libhdfs.so ->
> libhdfs.so.0.0.0
>
> lrwxrwxrwx 1 hduser hadoop      18 Feb  8  2015 libhadoop.so ->
> libhadoop.so.1.0.0
>
>
>
>
>
> And If I do a search for snappy I get
>
>
>
> hduser@rhes564::/home/hduser/hadoop-2.6.0> find ./ -name '*snappy*'
>
> ./share/hadoop/common/lib/snappy-java-1.0.4.1.jar
>
> ./share/hadoop/kms/tomcat/webapps/kms/WEB-INF/lib/snappy-java-1.0.4.1.jar
>
>
> ./share/hadoop/httpfs/tomcat/webapps/webhdfs/WEB-INF/lib/snappy-java-1.0.4.1.jar
>
> ./share/hadoop/mapreduce/lib/snappy-java-1.0.4.1.jar
>
> ./share/hadoop/tools/lib/snappy-java-1.0.4.1.jar
>
>
>
>
>
> Thanks,
>
>
>
> Dr Mich Talebzadeh
>
>
>
> LinkedIn * https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw
> <https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw>*
>
>
>
> *Sybase ASE 15 Gold Medal Award 2008*
>
> A Winning Strategy: Running the most Critical Financial Data on ASE 15
>
>
> http://login.sybase.com/files/Product_Overviews/ASE-Winning-Strategy-091908.pdf
>
> Author of the books* "A Practitioner’s Guide to Upgrading to Sybase ASE
> 15", ISBN 978-0-9563693-0-7*.
>
> co-author *"Sybase Transact SQL Guidelines Best Practices", ISBN
> 978-0-9759693-0-4*
>
> *Publications due shortly:*
>
> *Complex Event Processing in Heterogeneous Environments*, ISBN:
> 978-0-9563693-3-8
>
> *Oracle and Sybase, Concepts and Contrasts*, ISBN: 978-0-9563693-1-4, volume
> one out shortly
>
>
>
> http://talebzadehmich.wordpress.com
>
>
>
> NOTE: The information in this email is proprietary and confidential. This
> message is for the designated recipient only, if you are not the intended
> recipient, you should destroy it immediately. Any information in this
> message shall not be understood as given or endorsed by Peridale Technology
> Ltd, its subsidiaries or their employees, unless expressly so stated. It is
> the responsibility of the recipient to ensure that this email is virus
> free, therefore neither Peridale Ltd, its subsidiaries nor their employees
> accept any responsibility.
>
>
>
> *From:* Rajesh Balamohan [mailto:rajesh.balamohan@gmail.com]
> *Sent:* 05 January 2016 11:46
> *To:* user@hive.apache.org
> *Subject:* Re: Hive on TEZ fails starting
>
>
>
> Try '  beeline --hiveconf tez.task.launch.env="
> LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$HADOOP_COMMON_HOME/lib/native"
> --hiveconf tez.am.launch.env="LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$HADOOP_COMMON_HOME/lib/native"
> '.  Please check if you have the lib*.so available in the native folder (or
> point it to the folder which contains the so files)
>
>
>
> ~Rajesh.B
>
>
>
>
>
> On Tue, Jan 5, 2016 at 4:00 PM, Mich Talebzadeh <mich@peridale.co.uk>
> wrote:
>
> Hi,
>
>
>
> I have added the following to the LD_LIBRARY_PATH and JAVA_LIBRARY_PATH
>
>
>
> export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$HADOOP_COMMON_HOME/lib/native
>
> export JAVA_LIBRARY_PATH=$JAVA_LIBRARY_PATH:$HADOOP_COMMON_HOME/lib/native
>
>
>
> Trying to use TEZ, I still get the same error
>
>
>
> 0: jdbc:hive2://rhes564:10010/default> set hive.execution.engine=tez;
>
> No rows affected (0.002 seconds)
>
> 0: jdbc:hive2://rhes564:10010/default> use oraclehadoop;
>
> No rows affected (0.019 seconds)
>
> 0: jdbc:hive2://rhes564:10010/default> select count(1) from sales;
>
> INFO  : Tez session hasn't been created yet. Opening session
>
> INFO  :
>
>
>
> INFO  : Status: Running (Executing on YARN cluster with App id
> application_1451986680090_0002)
>
>
>
> INFO  : Map 1: -/-      Reducer 2: 0/1
>
> INFO  : Map 1: 0/1      Reducer 2: 0/1
>
> INFO  : Map 1: 0(+1)/1  Reducer 2: 0/1
>
> INFO  : Map 1: 0(+1,-1)/1       Reducer 2: 0/1
>
> INFO  : Map 1: 0(+1,-1)/1       Reducer 2: 0/1
>
> INFO  : Map 1: 0(+1,-2)/1       Reducer 2: 0/1
>
> INFO  : Map 1: 0(+1,-2)/1       Reducer 2: 0/1
>
> INFO  : Map 1: 0(+1,-3)/1       Reducer 2: 0/1
>
> INFO  : Map 1: 0(+1,-3)/1       Reducer 2: 0/1
>
>
>
> ERROR : Vertex failed, vertexName=Map 1,
> vertexId=vertex_1451986680090_0002_1_00, diagnostics=[Task failed,
> taskId=task_1451986680090_0002_1_00_000000, diagnostics=[TaskAttempt 0
> failed, info=[Error: Failure while running task:java.lang.RuntimeException:
> java.lang.UnsatisfiedLinkError:
> org.apache.hadoop.util.NativeCodeLoader.buildSupportsSnappy()Z
>
>
>
>
>
>
>
> Dr Mich Talebzadeh
>
>
>
> LinkedIn * https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw
> <https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw>*
>
>
>
> *Sybase ASE 15 Gold Medal Award 2008*
>
> A Winning Strategy: Running the most Critical Financial Data on ASE 15
>
>
> http://login.sybase.com/files/Product_Overviews/ASE-Winning-Strategy-091908.pdf
>
> Author of the books* "A Practitioner’s Guide to Upgrading to Sybase ASE
> 15", ISBN 978-0-9563693-0-7*.
>
> co-author *"Sybase Transact SQL Guidelines Best Practices", ISBN
> 978-0-9759693-0-4*
>
> *Publications due shortly:*
>
> *Complex Event Processing in Heterogeneous Environments*, ISBN:
> 978-0-9563693-3-8
>
> *Oracle and Sybase, Concepts and Contrasts*, ISBN: 978-0-9563693-1-4, volume
> one out shortly
>
>
>
> http://talebzadehmich.wordpress.com
>
>
>
> NOTE: The information in this email is proprietary and confidential. This
> message is for the designated recipient only, if you are not the intended
> recipient, you should destroy it immediately. Any information in this
> message shall not be understood as given or endorsed by Peridale Technology
> Ltd, its subsidiaries or their employees, unless expressly so stated. It is
> the responsibility of the recipient to ensure that this email is virus
> free, therefore neither Peridale Ltd, its subsidiaries nor their employees
> accept any responsibility.
>
>
>
> *From:* Rajesh Balamohan [mailto:rajesh.balamohan@gmail.com]
> *Sent:* 05 January 2016 00:35
> *To:* user@hive.apache.org
> *Subject:* Re: Hive on TEZ fails starting
>
>
>
> By default it should add "LD_LIBRARY_PATH" in the container (ref:
> https://github.com/apache/tez/blob/abfc8bfb0a8620d31697a31ad516674a8d3f9f7c/tez-api/src/main/java/org/apache/tez/dag/api/TezConfiguration.java#L358)
> . In case your snappy native libs are present elsewhere in the cluster
> deployment, you can override using "tez.task.launch.env" and
> "tez.am.launch.env" (refer:
> https://github.com/apache/tez/blob/abfc8bfb0a8620d31697a31ad516674a8d3f9f7c/tez-api/src/main/java/org/apache/tez/dag/api/TezConfiguration.java#L412
> )
>
>
>
> ~Rajesh.B
>
>
>
>
>
>
>
>
>
> --
>
> ~Rajesh.B
>

Mime
View raw message