hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Rick Hangartner <hangart...@strands.com>
Subject Re: [ANN] hbase-0.2.0 Release Candidate 1
Date Wed, 23 Jul 2008 17:40:29 GMT
Hi,

All in all, we like what we've seen of HBase-0.2.0 so far and the API  
changes.

Our upgrade test with Release Candidate 1 was a little bumpy but  
generally positive.

We set up a single virtualized test machine (in this case VMWare  
Fusion because an OS X machine was close at hand, although we also run  
Xen virtualized installs), with a virtual disk, running hadoop-0.16.4  
and hbase-0.1.3 with a small test table in a Debian install so we  
could rapidly test alternative upgrades paths and data migration.

We tried our own upgrade procedure once based on some previous  
experience and we tried the "formal" upgrade procedure  found at:

http://wiki.apache.org/hadoop/Hbase/HowToMigrate
http://wiki.apache.org/hadoop/Hadoop%20Upgrade

In each case, we of course started with a fresh copy of our  
virtualized machine.

Here's what we found:

1) In both approaches, the upgrade from hadoop-0.16.4 to hadoop-0.17.1  
seem to go OK.

2) In both approaches, when we tried to do the data migration from  
hbase-0.1.3 to hbase-0.2.0 we first got migration failures due to  
"unrecovered region server logs". Following the 'Redo Logs' comments  
in the "http://wiki.apache.org/hadoop/Hbase/HowToMigrate" doc,  
starting afresh with a  new copy of our virtualized system each time,  
we tried these methods of getting rid of the those logs and the fatal  
error:

    a) deleting just the log files in the "/hbase" directory
    b) deleting the entire contents of the "/hbase" directory (which  
means we lost our data, but we are just investigating the upgrade  
path, after all)
    c) deleting the "/hbase" directory entirely and creating a new "/ 
hbase" directory.

I should also note that we would need to repeat approach a) to be 100%  
certain of our results for that case.  (We've already repeated  
approaches b) and c) and have just run out of time for these upgrade  
tests because we need to get to other things).

In all cases, the migrate then failed as:

hbase@savory1:~/hbase$ bin/hbase migrate upgrade
08/07/22 18:03:16 INFO util.Migrate: Verifying that file system is
available...
08/07/22 18:03:16 INFO util.Migrate: Verifying that HBase is not  
running...
08/07/22 18:03:17 INFO ipc.Client: Retrying connect to server:
savory1/10.0.0.45:60000. Already tried 1 time(s).
  ...
08/07/22 18:03:26 INFO ipc.Client: Retrying connect to server:
savory1/10.0.0.45:60000. Already tried 10 time(s).
08/07/22 18:03:27 INFO util.Migrate: Starting upgrade
08/07/22 18:03:27 FATAL util.Migrate: Upgrade failed
java.io.IOException: Install 0.1.x of hbase and run its migration first
        at org.apache.hadoop.hbase.util.Migrate.run(Migrate.java:181)
        at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
        at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:79)
        at org.apache.hadoop.hbase.util.Migrate.main(Migrate.java:446)

As you can see, we were then in a bit of a Catch-22.  To re-install  
HBase-0.1.x also required re-installing Hadoop-0.16.4 (we tried  
reinstalling HBase-0.1.x without doing that!) so there was no way to  
proceed.  Attempting to start up HBase-0.2.0 just resulted in an error  
message that we needed to do the migrate.

3) Since this was just a test, we then blew away the disk used by  
Hadoop and re-built the namenode per a standard Hadoop new install.   
Hadoop-0.17.1 and Hbase-0.2.0 then started up just fine.  We only ran  
a few tests with the new Hbase command line shell in some ways we used  
the old HQL shell for sanity checks, and everything seems copacetic.

A few other comments:

- The new shell takes a bit of getting used to, but seems quite  
functional (we're not the biggest Ruby fans, but hey, someone took  
this on and upgraded the shell so we just say: Thanks!)

- We really like how timestamps have become first-class objects in the  
HBase-0.2.0 API .  Although, we were in the middle of developing some  
code under HBase-0.1.3 with workarounds for timestamps not being first- 
class objects and we will have to decide whether we should backup and  
re-develop for HBase-0.2.0 (we know we should), or plunge ahead with  
what we were doing under HBase-0.1.3 just to discard it in the near  
future because of the other advantages of HBase-0.2.0.  Is there  
anything we should consider in making this decision, perhaps about  
timing of any bug fixes and an official release of HBase-0.2.0?  
(HBase-0.2.1?).

Thanks for what continues to be a very useful and interesting product.

Rick

On Jul 22, 2008, at 3:24 PM, stack wrote:

> The first 0.2.0 release candidate is available for download:
>
> http://people.apache.org/~stack/hbase-0.2.0-candidate-1/
>
> Please take this release candidate for a spin. Check the  
> documentation, that unit tests all complete on your platform, etc.
>
> Should we release this candidate as hbase 0.2.0?  Vote yes or no  
> before Friday, July 25th.
>
> Release 0.2.0 has over 240 issues resolved [1] since the branch for  
> 0.1 hbase was made.  Be warned that hbase 0.2.0 is not backward  
> compatible with the hbase 0.1 API.  See [2] Izaak Rubins' notes on  
> the high-level API differences between 0.1 and 0.2.  For notes on  
> how to migrate your 0.1 era hbase data to 0.2, see Izaak's migration  
> guide [3].
>
> Yours,
> The HBase Team
>
> 1. https://issues.apache.org/jira/secure/ReleaseNote.jspa?version=12312955&styleName=Html&projectId=12310753&Create=Create
> 2. http://wiki.apache.org/hadoop/Hbase/Plan-0.2/APIChanges
> 3. http://wiki.apache.org/hadoop/Hbase/HowToMigrate


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message