hive-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Vinod Singh <vi...@vinodsingh.com>
Subject Re: HADOOP_HOME requirement
Date Wed, 18 Jul 2012 17:34:46 GMT
We are using Hive 0.7.1 and there  HADOOP_HOME must be exported so that it
is available as environment variable.

Thanks,
Vinod

On Wed, Jul 18, 2012 at 10:48 PM, Nitin Pawar <nitinpawar432@gmail.com>wrote:

> from hive trunk i can only see this
> I am not sure I am 100% sure but I remember setting up HADOOP_HOME always
>
>
> http://svn.apache.org/repos/asf/hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/MapRedTask.java
>
>
>       String hadoopExec = conf.getVar(HiveConf.ConfVars.HADOOPBIN);
>
> this change was introduced in 0.8
>
> from http://svn.apache.org/repos/asf/hive/branches/branch-0.9/common/src/java/org/apache/hadoop/hive/conf/HiveConf.java
<http://svn.apache.org/repos/asf/hive/branches/branch-0.8/common/src/java/org/apache/hadoop/hive/conf/HiveConf.java>
>
> HADOOPBIN("hadoop.bin.path", System.getenv("HADOOP_HOME") + "/bin/hadoop"),
>
> On Wed, Jul 18, 2012 at 10:38 PM, kulkarni.swarnim@gmail.com <
> kulkarni.swarnim@gmail.com> wrote:
>
>> 0.9
>>
>>
>> On Wed, Jul 18, 2012 at 12:04 PM, Nitin Pawar <nitinpawar432@gmail.com>wrote:
>>
>>> this also depends on what version of hive you are using
>>>
>>>
>>> On Wed, Jul 18, 2012 at 10:33 PM, kulkarni.swarnim@gmail.com <
>>> kulkarni.swarnim@gmail.com> wrote:
>>>
>>>> Thanks for your reply nitin.
>>>>
>>>> Ok. So you mean we always need to set HADOOP_HOME irrespective of
>>>> "hadoop" is on the path or not. Correct?
>>>>
>>>> Little confused because that contradicts what's mentioned here[1].
>>>>
>>>> [1]
>>>> https://cwiki.apache.org/confluence/display/Hive/GettingStarted#GettingStarted-RunningHive
>>>>
>>>>
>>>> Thanks,
>>>>
>>>> On Wed, Jul 18, 2012 at 11:59 AM, Nitin Pawar <nitinpawar432@gmail.com>wrote:
>>>>
>>>>> This is not a bug.
>>>>>
>>>>> even if hadoop was path, hive does not use it.
>>>>> Hive internally uses HADOOP_HOME in the code base. So you will always
>>>>> need to set that for hive.
>>>>> Where as for HADOOP clusters, HADOOP_HOME is deprecated but hive still
>>>>> needs it.
>>>>>
>>>>> Don't know if that answers your question
>>>>>
>>>>> Thanks,
>>>>> Nitin
>>>>>
>>>>>
>>>>> On Wed, Jul 18, 2012 at 10:01 PM, kulkarni.swarnim@gmail.com <
>>>>> kulkarni.swarnim@gmail.com> wrote:
>>>>>
>>>>>> Hello,
>>>>>>
>>>>>> The hive documentation states that either HADOOP_HOME should be set
>>>>>> or hadoop should be on the path. However for some cases, where HADOOP_HOME
>>>>>> was not set but hadoop was on path, I have seen this error pop up:
>>>>>>
>>>>>> java.io.IOException: *Cannot run program "null/bin/hadoop" *(in
>>>>>> directory "/root/swarnim/hive-0.9.0-cern1-SNAPSHOT"): java.io.IOException:
>>>>>> error=2, No such file or directory
>>>>>>  at java.lang.ProcessBuilder.start(ProcessBuilder.java:460)
>>>>>> at java.lang.Runtime.exec(Runtime.java:593)
>>>>>>  at java.lang.Runtime.exec(Runtime.java:431)
>>>>>> at
>>>>>> org.apache.hadoop.hive.ql.exec.MapRedTask.execute(MapRedTask.java:268)
>>>>>>  at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:134)
>>>>>> at
>>>>>> org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:57)
>>>>>>  at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:1326)
>>>>>> at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1118)
>>>>>>  at org.apache.hadoop.hive.ql.Driver.run(Driver.java:951)
>>>>>> at
>>>>>> org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:258)
>>>>>>  at
>>>>>> org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:215)
>>>>>> at
>>>>>> org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:406)
>>>>>>  at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:689)
>>>>>> at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:557)
>>>>>>  at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>>>>>> at
>>>>>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>>>>>>  at
>>>>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>>>>>> at java.lang.reflect.Method.invoke(Method.java:597)
>>>>>>  at org.apache.hadoop.util.RunJar.main(RunJar.java:208)
>>>>>>
>>>>>> Digging into the code in MapRedTask.java, I found the following
>>>>>> (simplified):
>>>>>>
>>>>>> String *hadoopExec* = conf.getVar(System.getenv("HADOOP_HOME") +
>>>>>> "/bin/hadoop");
>>>>>> ...
>>>>>>
>>>>>> Runtime.getRuntime().exec(*hadoopExec*, env, new File(workDir));
>>>>>>
>>>>>> Clearly, if HADOOP_HOME is not set, the command that it would try
to
>>>>>> execute is "null/bin/hadoop" which is exactly the exception I am
getting.
>>>>>>
>>>>>> Has anyone else run into this before? Is this a bug?
>>>>>>
>>>>>> Thanks,
>>>>>> --
>>>>>> Swarnim
>>>>>>
>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> Nitin Pawar
>>>>>
>>>>>
>>>>
>>>>
>>>> --
>>>> Swarnim
>>>>
>>>
>>>
>>>
>>> --
>>> Nitin Pawar
>>>
>>>
>>
>>
>> --
>> Swarnim
>>
>
>
>
> --
> Nitin Pawar
>
>

Mime
View raw message