hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From larsgeorge <l...@worldlingo.com>
Subject Re: Question for HBase users
Date Sat, 05 Jan 2008 06:01:16 GMT

Jim,

I have inserted about 5million documents into HBase and translate them into
15 languages (means I end up with about 75million in the end). That data is
only recreatable if we process them costly again. So I am in need of a
migration path. 

For me this is a definitely +1 for a migration tool.

Sorry to be a hassle like this. :\

Lars

----
Lars George, CTO
WorldLingo


Jim Kellerman wrote:
> 
> Do you have data stored in HBase that you cannot recreate?
> 
> HADOOP-2478 will introduce an incompatible change in how HBase
> lays out files in HDFS so that should the root or meta tables
> be corrupted, it will be possible to reconstruct them from
> information in the file system alone.
> 
> The problem is in building a migration utility. Anything that
> we could build to migrate from the current file structure to
> the new file structure would require that the root an meta
> regions be absolutely correct. If they are not, the migration
> would fail, because there is not enough information on disk
> currently to rebuild the root and meta regions.
> 
> Is it acceptable for this change to be made without the provision
> of an upgrade utility?
> 
> If not, are you willing to accept the risk that the upgrade
> may fail if you have corruption in your root or meta regions?
> 
> After HADOOP-2478, we will be able to build a fault tolerant
> upgrade utility, should HBase's file structure change again.
> Additionally, we will be able to provide the equivalent of
> fsck for HBase after HADOOP-2478.
> 
> ---
> Jim Kellerman, Senior Engineer; Powerset
> 
> No virus found in this outgoing message.
> Checked by AVG Free Edition.
> Version: 7.5.516 / Virus Database: 269.17.13/1207 - Release Date: 1/2/2008
> 11:29 AM
> 
> 
> 

-- 
View this message in context: http://www.nabble.com/Question-for-HBase-users-tp14607732p14630344.html
Sent from the Hadoop Users mailing list archive at Nabble.com.


Mime
View raw message