hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Roman Shaposhnik (Commented) (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-5286) bin/hbase's logic of adding Hadoop jar files to the classpath is fragile when presented with split packaged Hadoop 0.23 installation
Date Wed, 01 Feb 2012 22:47:58 GMT

    [ https://issues.apache.org/jira/browse/HBASE-5286?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13198285#comment-13198285
] 

Roman Shaposhnik commented on HBASE-5286:
-----------------------------------------

@stack, sed-magic is certainly possible, but it will be quite difficult to support all the
layouts of hadoop in a single hbase script and account for all the naming conventions

How about if we use hadoop binary to execute hbase (when we can find it) and fall back on
standalone execution when we can't. Sort of what Pig does over here (line #234): 
   http://svn.apache.org/viewvc/pig/trunk/bin/pig?revision=1228547&view=markup
                
> bin/hbase's logic of adding Hadoop jar files to the classpath is fragile when presented
with split packaged Hadoop 0.23 installation
> ------------------------------------------------------------------------------------------------------------------------------------
>
>                 Key: HBASE-5286
>                 URL: https://issues.apache.org/jira/browse/HBASE-5286
>             Project: HBase
>          Issue Type: Bug
>          Components: scripts
>    Affects Versions: 0.92.0
>            Reporter: Roman Shaposhnik
>            Assignee: Roman Shaposhnik
>
> Here's the bit from bin/hbase that might need TLC now that Hadoop can be spotted in the
wild in split-package configuration:
> {noformat}
> #If avail, add Hadoop to the CLASSPATH and to the JAVA_LIBRARY_PATH
> if [ ! -z $HADOOP_HOME ]; then
>   HADOOPCPPATH=""
>   if [ -z $HADOOP_CONF_DIR ]; then
>     HADOOPCPPATH=$(append_path "${HADOOPCPPATH}" "${HADOOP_HOME}/conf")
>   else
>     HADOOPCPPATH=$(append_path "${HADOOPCPPATH}" "${HADOOP_CONF_DIR}")
>   fi
>   if [ "`echo ${HADOOP_HOME}/hadoop-core*.jar`" != "${HADOOP_HOME}/hadoop-core*.jar"
] ; then
>     HADOOPCPPATH=$(append_path "${HADOOPCPPATH}" `ls ${HADOOP_HOME}/hadoop-core*.jar
| head -1`)
>   else
>     HADOOPCPPATH=$(append_path "${HADOOPCPPATH}" `ls ${HADOOP_HOME}/hadoop-common*.jar
| head -1`)
>     HADOOPCPPATH=$(append_path "${HADOOPCPPATH}" `ls ${HADOOP_HOME}/hadoop-hdfs*.jar
| head -1`)
>     HADOOPCPPATH=$(append_path "${HADOOPCPPATH}" `ls ${HADOOP_HOME}/hadoop-mapred*.jar
| head -1`)
>   fi
> {noformat}
> There's a couple of issues with the above code:
>    0. HADOOP_HOME is now deprecated in Hadoop 0.23
>    1. the list of jar files added to the class-path should be revised
>    2. we need to figure out a more robust way to get the jar files that are needed to
the classpath (things like hadoop-mapred*.jar tend to match src/test jars as well)
> Better yet, it would be useful to look into whether we can transition HBase's bin/hbase
onto using bin/hadoop as a launcher script instead of direct JAVA invocations (Pig, Hive,
Sqoop and Mahout already do that)

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message