hadoop-common-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Apache Wiki <wikidi...@apache.org>
Subject [Hadoop Wiki] Update of "Hive/HBaseIntegration" by JohnSichi
Date Mon, 14 Feb 2011 21:41:08 GMT
Dear Wiki user,

You have subscribed to a wiki page or wiki category on "Hadoop Wiki" for change notification.

The "Hive/HBaseIntegration" page has been changed by JohnSichi.
http://wiki.apache.org/hadoop/Hive/HBaseIntegration?action=diff&rev1=33&rev2=34

--------------------------------------------------

  == Usage ==
  
  The storage handler is built as an independent module,
- {{{hive_hbase_handler.jar}}}, which must be available on the Hive
+ {{{hive-hbase-handler-x.y.z.jar}}}, which must be available on the Hive
  client auxpath, along with HBase and Zookeeper jars.  It also requires
  the correct configuration property to be set in order to connect to the
  right HBase master.  See [[http://hadoop.apache.org/hbase/docs/current/api/overview-summary.html#overview_description|the
HBase documentation]] for how to set up an HBase cluster.
  
- Here's an example using CLI from a source build environment, targeting a single-node HBase
server:
+ Here's an example using CLI from a source build environment, targeting a single-node HBase
server.  (Note that the jar locations and names have changed in Hive 0.7, so for earlier releases,
some changes are needed.)
  
  {{{
- $HIVE_SRC/build/dist/bin/hive --auxpath $HIVE_SRC/build/dist/lib/hive_hbase-handler.jar,$HIVE_SRC/build/dist/lib/hbase-0.20.3.jar,$HIVE_SRC/hbase-handler/build/dist/lib/zookeeper-3.2.2.jar
-hiveconf hbase.master=hbase.yoyodyne.com:60000
+ $HIVE_SRC/build/dist/bin/hive --auxpath $HIVE_SRC/build/dist/lib/hive-hbase-handler-0.7.0.jar,$HIVE_SRC/build/dist/lib/hbase-0.89.0-SNAPSHOT.jar,$HIVE_SRC/build/dist/lib/zookeeper-3.3.1.jar
-hiveconf hbase.master=hbase.yoyodyne.com:60000
  }}}
  
  Here's an example which instead targets a distributed HBase cluster where a quorum of 3
zookeepers is used to elect the HBase master:
  
  {{{
- $HIVE_SRC/build/dist/bin/hive --auxpath $HIVE_SRC/build/dist/lib/hive_hbase-handler.jar,$HIVE_SRC/build/dist/lib/hbase-0.20.3.jar,$HIVE_SRC/build/dist/lib/zookeeper-3.2.2.jar
-hiveconf hbase.zookeeper.quorum=zk1.yoyodyne.com,zk2.yoyodyne.com,zk3.yoyodyne.com
+ $HIVE_SRC/build/dist/bin/hive --auxpath $HIVE_SRC/build/dist/lib/hive-hbase-handler-0.7.0.jar,$HIVE_SRC/build/dist/lib/hbase-0.89.0-SNAPSHOT.jar,$HIVE_SRC/build/dist/lib/zookeeper-3.3.1.jar
-hiveconf hbase.zookeeper.quorum=zk1.yoyodyne.com,zk2.yoyodyne.com,zk3.yoyodyne.com
  }}}
  
  The handler requires Hadoop 0.20 or higher, and has only been tested
- with dependency versions hadoop-0.20.0, hbase-0.20.3 and zookeeper-3.2.2.  If you are not
using hbase-0.20.3, you will need to rebuild the handler with the HBase jar matching your
version, and change the --auxpath above accordingly.  Failure to use matching versions will
lead to misleading connection failures such as MasterNotRunningException since the HBase RPC
protocol changes often.
+ with dependency versions hadoop-0.20.x, hbase-0.89.0 and zookeeper-3.3.1.  If you are not
using hbase-0.89.0, you will need to rebuild the handler with the HBase jar matching your
version, and change the --auxpath above accordingly.  Failure to use matching versions will
lead to misleading connection failures such as MasterNotRunningException since the HBase RPC
protocol changes often.
  
  In order to create a new HBase table which is to be managed by Hive,
  use the STORED BY clause on CREATE TABLE:
@@ -323, +323 @@

  == Build ==
  
  Code for the storage handler is located under
+ {{{hive/trunk/hbase-handler}}}.
- {{{hive/trunk/hbase-handler}}}.  The Hive build automatically enables
- the storage handler build for {{{hadoop.version=0.20.x}}}, and
- disables it for any other Hadoop version.  This behavior can be
- overridden by setting ant property {{{hbase.enabled}}} to either
- {{{true}}} or {{{false}}}.
  
- HBase and Zookeeper dependencies are currently checked in under 
+ HBase and Zookeeper dependencies are fetched via ivy.
- {{{hbase-handler/lib}}}.  We will convert this to use Ivy instead once
- the corresponding POM's are available.
  
  == Tests ==
  

Mime
View raw message