hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Zhihong Yu (Commented) (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-5504) Online Merge
Date Fri, 02 Mar 2012 01:58:59 GMT

    [ https://issues.apache.org/jira/browse/HBASE-5504?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13220601#comment-13220601
] 

Zhihong Yu commented on HBASE-5504:
-----------------------------------

w.r.t. the comment @ 02/Mar/12 01:18,
Is it possible that there would be more than one master-coordinated task for the same table
running at the same time ?

If so, what should the state store ?

I think we should remove the noob label for this JIRA.
                
> Online Merge
> ------------
>
>                 Key: HBASE-5504
>                 URL: https://issues.apache.org/jira/browse/HBASE-5504
>             Project: HBase
>          Issue Type: Brainstorming
>          Components: client, master, shell, zookeeper
>    Affects Versions: 0.94.0
>            Reporter: Mubarak Seyed
>              Labels: noob
>             Fix For: 0.96.0
>
>
> As discussed, please refer the discussion at [HBASE-4991|https://issues.apache.org/jira/browse/HBASE-4991]
> Design suggestion from Stack:
> {quote}
> I suggest a design below. It has some prerequisites, some general function that this
feature could use (and others). The prereqs if you think them good, could be done outside
of this JIRA.
> Here's a suggested rough outline of how I think this feature should run. The feature
I'm describing below is merge and deleteRegion for I see them as in essence the same thing.
> (C) Client, (M) Master, RS (Region server), ZK (ZooKeeper)
> 1. Client calls merge or deleteRegion API. API is a range of rows. (C)
> 2. Master gets call. (M)
> 3. Master obtains a write lock on table so it can't be disabled from under us. The write
lock will also disable splitting. This is one of the prereqs I think. Its HBASE-5494 (Or we
could just do something simpler where we have a flag up in zk that splitRegion checks but
thats less useful I think; OR we do the dynamic configs issue and set splits to off via a
config. change). There'd be a timer for how long we wait on the table lock. (M -> ZK)
> 4. If we get the lock, write intent to merge a range up into zk. It also hoists into
zk if its a pure merge or a merge that drops the region data (a deleteRegion call) (M)
> 5. Return to the client either our failed attempt at locking the table or an id of some
sort used identifying this running operation; can use it querying status. (M -> C)
> 6. Turn off balancer. TODO/prereq: Do it in a way that is persisted. Balancer switch
currently in memory only so if master crashes, new master will come up in balancing mode #
(If we had dynamic config. could hoist up to zk a config. that disables the balancer rather
than have a balancer-specific flag/znode OR if a write lock outstanding on a table, then the
balancer does not balance regions in the locked table - this latter might be the easiest to
do) (M)
> 7. Write into zk that just turned off the balancer (If it was on) (M -> ZK)
> 8. Get regions that are involved in the span (M)
> 9. Hoist the list up into zk. (M -> ZK)
> 10. Create region to span the range. (M)
> 11. Write that we did this up into zk. (M -> ZK)
> 12. Close regions in parallel. Confirm close in parallel. (M -> RS)
> 13. Write up into zk regions closed (This might not be necessary since can ask if region
is open). (M -> ZK)
> 14. If a merge and not a delete region, move files under new region. Might multithread
this (moves should go pretty fast). If a deleteregion, we skip this step. (M)
> 15. On completion mark zk (though may not be necessary since its easy to look in fs to
see state of move). (M -> ZK)
> 16. Edit .META. (M)
> 17. Confirm edits went in. (M)
> 18. Move old regions to hbase trash folder TODO: There is no trash folder under /hbase
currently. We should add one. (M)
> 19. Enable balancer (if it was off) (M)
> 20. Unlock table (M)
> {quote}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message