hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Harsh J <ha...@cloudera.com>
Subject Re: Why $HADOOP_PREFIX ?
Date Wed, 01 Feb 2012 17:08:03 GMT
Personal opinion here: For branch-1, I do think the earlier tarball
structure was better. I do not see why it had to change for this
version at least. Possibly was changed during all the work of adding
packaging-related scripts for rpm/deb into Hadoop itself, but the
tarball right now is not as usable as was before, and the older format
would've still worked today.

On Wed, Feb 1, 2012 at 10:31 PM, Robert Evans <evans@yahoo-inc.com> wrote:
> I think it comes down to a long history of splitting and then remerging the hadoop project.
 I could be wrong about a lot of this so take it worth a grain of salt.  Hadoop originally,
and still is on 1.0 a single project.  HDFS, mapreduce and common are all compiled together
into a single jar hadoop-core.  In that respect HADOOP_HOME made a lot of since because it
was a single thing, with some dependencies that needed to be found by some shell scripts.
> Fast forward the projects were split, HADOOP_HOME was deprecated, and HADOOP_COMMON_HOME,
HADOOP_MAPRED_HOME, and HADOOP_HDFS_HOME were born.  But if we install them all into a single
tree it is a pain to configure all of these to point to the same place, but HADOOP_HOME is
deprecated, so HADOOP_PREFIX was born.  NOTE: like was stated before all of these are supposed
to be hidden from the end user and are intended more towards packaging and deploying hadoop.
 Also the process is not done and it is likely to change further.
> --Bobby Evans
> On 2/1/12 8:10 AM, "praveenesh kumar" <praveenesh@gmail.com> wrote:
> Interesting and strange.
> but are there any reason for setting $HADOOP_HOME to $HADOOP_PREFIX in
> hadoop-conf.sh
> and then checking in /bin/hadoop.sh whether $HADOOP_HOME is not equal to ""
> I mean if I comment out the export HADOOP_HOME=${HADOOP_PREFIX} in
> hadoop-conf.sh, does it make any difference ?
> Thanks,
> Praveenesh
> On Wed, Feb 1, 2012 at 6:04 PM, Prashant Sharma <prashant.s@imaginea.com>wrote:
>> I think you have misunderstood something. AFAIK or understand  these
>> variables are set automatically when you run a script. it's name is obscure
>> for some strange reason. ;).
>> Warning: $HADOOP_HOME is deprecated is always there. whether the variable
>> is set or not. Why?
>> Because the hadoop-config is sourced in all scripts. And all it does is
>> sets HADOOP_PREFIX as HADOOP_HOME. I think this can be reported as a bug.
>> -P
>> On Wed, Feb 1, 2012 at 5:46 PM, praveenesh kumar <praveenesh@gmail.com
>> >wrote:
>> > Does anyone have idea on Why $HADOOP_PREFIX was introduced instead of
>> > $HADOOP_HOME in hadoop 0.20.205 ?
>> >
>> > I believe $HADOOP_HOME was not giving any troubles or is there a
>> reason/new
>> > feature that require $HADOOP_PREFIX to be added ?
>> >
>> > Its a kind of funny, but I got habitual of using $HADOOP_HOME. Just
>> curious
>> > to know for this change.
>> > Also, there are some old packages ( I am not referring apache/cloudera/or
>> > any hadoop distribution ), that depends on hadoop that still uses
>> > $HADOOP_HOME inside. So its kind of weird when you use those packages,
>> you
>> > still get warning messages even though its suppressed from Hadoop side.
>> >
>> >
>> > Thanks,
>> > Praveenesh
>> >

Harsh J
Customer Ops. Engineer
Cloudera | http://tiny.cloudera.com/about

View raw message