hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "dhruba borthakur (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HBASE-2231) Compaction events should be written to HLog
Date Fri, 04 Mar 2011 20:45:46 GMT

    [ https://issues.apache.org/jira/browse/HBASE-2231?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13002801#comment-13002801
] 

dhruba borthakur commented on HBASE-2231:
-----------------------------------------

yet another approach would be the following:

the hdfs files remain inside a path that has the currently assigned regionname encoded in
it. For example, a path of the form /hbase/tablename/regionname/currently-assigned-regionserver/....

when a master assigns a region to a regionserver, it makes the change in zookeeper and then
issues one rename call to hdfs to rename the top level region directory name to the appropriate
regionserver name. In some ways, this is similar to IO-fencing in the sense that any old region
servers cannot operate on any of the existing files of this region. Recovery from a master
restart and/or region-server restart should be simple, and I can list those steps later.

This has some drawbacks, especially since this makes the region data have variable path names
at different points in time, could cause some issues in bulk-loading via map-reduce.


> Compaction events should be written to HLog
> -------------------------------------------
>
>                 Key: HBASE-2231
>                 URL: https://issues.apache.org/jira/browse/HBASE-2231
>             Project: HBase
>          Issue Type: Improvement
>          Components: regionserver
>            Reporter: Todd Lipcon
>            Assignee: Todd Lipcon
>            Priority: Blocker
>              Labels: moved_from_0_20_5
>             Fix For: 0.92.0
>
>         Attachments: hbase-2231-testcase.txt, hbase-2231.txt
>
>
> The sequence for a compaction should look like this:
> # Compact region to "new" files
> # Write a "Compacted Region" entry to the HLog
> # Delete "old" files
> This deals with a case where the RS has paused between step 1 and 2 and the regions have
since been reassigned.

-- 
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message