whirr-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Bruno Dumon (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (WHIRR-240) [HBase] Enable support for HBase 0.90.x
Date Mon, 18 Jul 2011 11:05:00 GMT

    [ https://issues.apache.org/jira/browse/WHIRR-240?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13066888#comment-13066888
] 

Bruno Dumon commented on WHIRR-240:
-----------------------------------

I had not arrived at trying the integration tests before (sorry), I'm looking into it now.

I identified the following problems:

* whirr.instance-templates: the thriftserver was run on a node on which there is no hadoop.
This does not work as then we can't copy over the hadoop-core jar. This will also be a problem
for WHIRR-339 (hbase-site.xml generation), I'll add a comment over there. The 'solution' is
to adjust the template, I think this is fair enough since there are other limitations too
on how templates must be structured.

* Whirr decides to first configure the node with the regionserver, and after that the one
with zookeeper. This fails because the regionserver will only wait/retry a limited amount
of time to connect to ZooKeeper on startup (20s, I checked in the logs that zookeeper actually
started after the regionserver exited, about 10s later). Luckily, there is a property to control
this: hbase.zookeeper.recoverable.waittime. I put it to 5 minutes (this prop only has effect
on startup AFAICS).

With these changes, the test runs successfully for me.

To make the tests work for both 0.89 and 0.90, I had to add an additional test class and properties
file, the name of the properties file is passed to HBaseServiceController.getInstance(). Let
me know if you prefer to have it done some other way.

Note that, in services/hbase/pom.xml, I left the hbase.version property on 0.89etcetera which
does work for both since it uses the thrift interface which seems to be compatible.

The tests for 0.89 still run, but since copying hadoop-core is done regardless of the HBase
version, the above mentioned limitation of needing to have Hadoop installed on the node with
thrift now also applies to 0.89.

The hbase.zookeeper.recoverable.waittime is specified in the cluster configuration, which
depends on WHIRR-339. Without WHIRR-339, it needs to be hardcoded in the configure_(cdh_)hbase.sh

Through the properties passed with -Dconfig to the integration test, I supplied the following:

whirr.image-id=eu-west-1/ami-619ea915 (canonical 11.04 instance store EU)
whirr.hardware-id=m1.large
whirr.location-id=eu-west-1a

> [HBase] Enable support for HBase 0.90.x
> ---------------------------------------
>
>                 Key: WHIRR-240
>                 URL: https://issues.apache.org/jira/browse/WHIRR-240
>             Project: Whirr
>          Issue Type: New Feature
>          Components: service/hbase
>    Affects Versions: 0.3.0
>            Reporter: Lars George
>         Attachments: WHIRR-240-tests.patch, WHIRR-240.patch, hbase-ec2-090.properties
>
>
> HBase 0.90.0 is a difficult release as it either needs CDH or a patched Hadoop (with
append) to work. The Apache tarballs won't do and HBase will not start. 
> One way possible is to deploy the Apache Hadoop 0.20.2 tarball and then override the
core jar with the one supplied by HBase. Since HBase relies on Hadoop to be setup by the Whirr
service we would need some surgery that would imply service ordering. 

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message