hadoop-common-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Apache Wiki <wikidi...@apache.org>
Subject [Hadoop Wiki] Update of "Hbase/Migration" by stack
Date Fri, 18 Jan 2008 02:52:25 GMT
Dear Wiki user,

You have subscribed to a wiki page or wiki category on "Hadoop Wiki" for change notification.

The following page has been changed by stack:

The comment on the change is:
Initial commit

New page:
A working document to figure how migrations will work in hbase.  Initial outline comes of
a trawl of the content of [https://issues.apache.org/jira/browse/HADOOP-2394 HADOOP-2394].
 Does not consider hadoop migrations.

== Assertions ==

 * All hbase data and state is out on the FileSystem: Moving from one version should be just
a case of moving or rewriting files on the FileSystem.
 * Hbase cannot be running when a migration is run.
 * Sometimes, the amount of on-filesystem data that needs to be changed will be large so migration
will need to run a MR job.
 * hbase FS image needs versioning.  On startup, hbase will check the FS version.  If awry,
hbase will shut itself down emitting a migration needed message.  Versions are finer-grained
than release number (svn revision?).
 * The commit of every incompatible change would be accompanied by a script that can move
hbase across the incompatibility.
 * A migration runs migration scripts in order, from oldest through to latest (Migration scripts
are named in a manner that dictates an order -- or a catalog file lists the order in which
scripts are run).
 * Downtime must be minimal.
 * Migration script will do no damage if run when there is nothing to migrate

== Dependencies ==
 * Hbase fast backup to be run before migration to protect against data loss

== Issues ==
 * Should hbase classes be versioned and know how to migrate themselves?  IMO, seems like
excessive overhead especially for smaller classes HStoreKey and its like.  If not, how to
go between versions (How to float two versions of same class in same job?).

View raw message