hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Bryan Duxbury <br...@rapleaf.com>
Subject Re: [jira] Commented: (HADOOP-2394) Add supprt for migrating between hbase versions
Date Fri, 14 Dec 2007 21:58:36 GMT
The scheme you propose would be good so long as we only ever do  
things like rename files and move them around. If we ever decide to  
change something significant, like the underlying file structure  
(like if we break away from using MapFile or something), then we'd  
need the ability to read the old version as well as write the new  
ones. What would you like to be able to do in these instances?

On Dec 14, 2007, at 12:46 PM, stack (JIRA) wrote:

>
>     [ https://issues.apache.org/jira/browse/HADOOP-2394? 
> page=com.atlassian.jira.plugin.system.issuetabpanels:comment- 
> tabpanel#action_12551938 ]
>
> stack commented on HADOOP-2394:
> -------------------------------
>
> I like the rails idea.  Migration should support going in both  
> directions I'd say.
>
> hbase state is all kept out in the filesystem so hopefully,  
> filesystem machinations should be all thats required making  
> migrations.
>
> HStoreFiles are MapFiles + an info file stored in a sympathetic  
> directory.  This info file has little in it currently -- just  
> sequence id.  Could also have hbase version.  For log files,  
> perhaps first record is stamp of the hbase version doing the writing.
>
> It occurred to me that migrations could entail significant  
> rewriting of on-filesystem data.   To distribute the migration, we  
> could we could have the master and regionservers run the  
> migrations.  Each server on startup would look for any migrations  
> to run and just run them if any found.  Nice thing about this is  
> that we'd get the migration job distributed.   But thinking on it,  
> probably better to have the migration done outside of hbase in its  
> own dedicated MR job.  Would be easier tracking failures and  
> running reversals.
>
>> Add supprt for migrating between hbase versions
>> -----------------------------------------------
>>
>>                 Key: HADOOP-2394
>>                 URL: https://issues.apache.org/jira/browse/ 
>> HADOOP-2394
>>             Project: Hadoop
>>          Issue Type: Improvement
>>          Components: contrib/hbase
>>            Reporter: Johan Oskarsson
>>
>> If Hbase is to be used to serve data to live systems we would need  
>> a way to upgrade both the underlying hadoop installation and hbase  
>> to newer versions with minimal downtime.
>
> -- 
> This message is automatically generated by JIRA.
> -
> You can reply to this email to add a comment to the issue online.
>


Mime
View raw message