hadoop-common-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Apache Wiki <wikidi...@apache.org>
Subject [Hadoop Wiki] Update of "Hbase/MapReduce" by stack
Date Fri, 29 Feb 2008 04:18:05 GMT
Dear Wiki user,

You have subscribed to a wiki page or wiki category on "Hadoop Wiki" for change notification.

The following page has been changed by stack:
http://wiki.apache.org/hadoop/Hbase/MapReduce

The comment on the change is:
Minor edit: add conf dir.

------------------------------------------------------------------------------
  
  !MapReduce jobs deployed to a mapreduce cluster do not by default have access to the hbase
configuration under ''$HBASE_CONF_DIR'' nor to hbase classes.
  
- You could add ''hbase-site.xml'' to $HADOOP_HOME/conf and add hbase.jar to the $HADOOP_HOME/lib
and copy these changes across your cluster but he cleanest means of adding hbase configuration
and classes to the cluster CLASSPATH is by uncommenting ''HADOOP_CLASSPATH'' in ''$HADOOP_HOME/conf/hadoop-env.sh''
and adding the path to the hbase jar and ''$HBASE_CONF_DIR'' directory.  Then copy the amended
configuration across the cluster.  You'll need to restart the mapreduce cluster if you want
it to notice the new configuration.
+ You could add ''hbase-site.xml'' to $HADOOP_HOME/conf and add hbase.jar to the $HADOOP_HOME/lib
and copy these changes across your cluster but the cleanest means of adding hbase configuration
and classes to the cluster CLASSPATH is by uncommenting ''HADOOP_CLASSPATH'' in ''$HADOOP_HOME/conf/hadoop-env.sh''
and adding the path to the hbase jar and ''$HBASE_CONF_DIR'' directory.  Then copy the amended
configuration around the cluster.  You'll probably need to restart the mapreduce cluster if
you want it to notice the new configuration (You may not have to).
  
- For example, here is how you would amend ''hadoop-env.sh'' adding hbase classes and the
!PerformanceEvaluation class from hbase test classes to the hadoop ''CLASSPATH'':
+ For example, here is how you would amend ''hadoop-env.sh'' adding the hbase jar, conf, and
the !PerformanceEvaluation class from hbase test classes to the hadoop ''CLASSPATH'':
  
  {{{# Extra Java CLASSPATH elements.  Optional.
  # export HADOOP_CLASSPATH=
- export HADOOP_CLASSPATH=$HBASE_HOME/build/test:$HBASE_HOME/build/hadoop-0.15.0-dev-hbase.jar}}}
+ export HADOOP_CLASSPATH=$HBASE_HOME/build/test:$HBASE_HOME/build/hadoop-0.16.0-hbase.jar:$HBASE_HOME/conf}}}
  
  Expand $HBASE_HOME appropriately in the in accordance with your local environment
  
  And then, this is how you would run the PerformanceEvaluation MR job to put up 4 clients:
  
- {{{ > $HADOOP_HOME/bin/hadoop org.apache.hadoop.hbase.PerformanceEvaluation sequentialWrite
4
+ {{{ > $HADOOP_HOME/bin/hadoop org.apache.hadoop.hbase.PerformanceEvaluation sequentialWrite
4}}}
- }}}
  
  The PerformanceEvaluation class wil be found on the CLASSPATH because you added $HBASE_HOME/build/test
to HADOOP_CLASSPATH
  

Mime
View raw message