hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "jiraposter@reviews.apache.org (Commented) (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-4797) [availability] Skip recovered.edits files with edits we know older than what region currently has
Date Tue, 22 Nov 2011 04:58:41 GMT

    [ https://issues.apache.org/jira/browse/HBASE-4797?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13154877#comment-13154877
] 

jiraposter@reviews.apache.org commented on HBASE-4797:
------------------------------------------------------


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/2906/#review3425
-----------------------------------------------------------


+1 on patch

- ramkrishna


On 2011-11-22 01:02:17, Jimmy Xiang wrote:
bq.  
bq.  -----------------------------------------------------------
bq.  This is an automatically generated e-mail. To reply, visit:
bq.  https://reviews.apache.org/r/2906/
bq.  -----------------------------------------------------------
bq.  
bq.  (Updated 2011-11-22 01:02:17)
bq.  
bq.  
bq.  Review request for hbase, Todd Lipcon and Michael Stack.
bq.  
bq.  
bq.  Summary
bq.  -------
bq.  
bq.  If there are multiple recovered edits files, I used the file name to find the initial
sequence id.  After these files are sorted, we can find a file's possible maximum sequence
id based on the next file's initial sequence id.  If the maximum sequence id is smaller than
the current sequence id, the whole recovered edits file is old and ignored.
bq.  
bq.  
bq.  This addresses bug HBASE-4797.
bq.      https://issues.apache.org/jira/browse/HBASE-4797
bq.  
bq.  
bq.  Diffs
bq.  -----
bq.  
bq.    src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java 8b89661 
bq.    src/test/java/org/apache/hadoop/hbase/regionserver/TestHRegion.java 5daa02b 
bq.  
bq.  Diff: https://reviews.apache.org/r/2906/diff
bq.  
bq.  
bq.  Testing
bq.  -------
bq.  
bq.  Added test case to TestHRegion, and all the tests in this test are passed.
bq.  
bq.  
bq.  Thanks,
bq.  
bq.  Jimmy
bq.  
bq.


                
> [availability] Skip recovered.edits files with edits we know older than what region currently
has
> -------------------------------------------------------------------------------------------------
>
>                 Key: HBASE-4797
>                 URL: https://issues.apache.org/jira/browse/HBASE-4797
>             Project: HBase
>          Issue Type: Bug
>          Components: performance
>            Reporter: stack
>            Assignee: Jimmy Xiang
>            Priority: Critical
>              Labels: noob
>             Fix For: 0.94.0
>
>         Attachments: 0001-HBASE-4797-[availability]-skip-older-edits.patch, 0001-HBASE-4797-[availability]-skip-older-edits.patch
>
>
> Testing 0.92, I crashed all servers out.  Another bug makes it so WALs are not getting
cleaned so I had 7000 regions to replay.  The distributed split code did a nice job and cluster
came back but interesting is that some hot regions ended up having loads of recovered.edits
files -- tens if not hundreds -- to replay against the region (can we bulk load recovered.edits
instead of replaying them?).  Each recovered.edits file is taking about a second to process
(though only about 30 odd edits per file it seems).  The region is unavailable during this
time.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message