hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "jiraposter@reviews.apache.org (Commented) (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-4377) [hbck] Offline rebuild .META. from fs data only.
Date Mon, 03 Oct 2011 18:17:41 GMT

    [ https://issues.apache.org/jira/browse/HBASE-4377?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13119461#comment-13119461
] 

jiraposter@reviews.apache.org commented on HBASE-4377:
------------------------------------------------------



bq.  On 2011-09-30 21:27:16, Ted Yu wrote:
bq.  > src/main/java/org/apache/hadoop/hbase/master/MasterFileSystem.java, line 376
bq.  > <https://reviews.apache.org/r/2126/diff/1/?file=46564#file46564line376>
bq.  >
bq.  >     b is not needed here, same with question mark.
bq.  
bq.  jmhsieh wrote:
bq.      k

javadoc form for @param is to list the parameter name, so it should be there.  agree that
no question mark should be there.  i think the javadoc-y phrasing would be something like
"whether to enable in-memory caching or not"


- Jonathan


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/2126/#review2231
-----------------------------------------------------------


On 2011-09-30 00:02:16, jmhsieh wrote:
bq.  
bq.  -----------------------------------------------------------
bq.  This is an automatically generated e-mail. To reply, visit:
bq.  https://reviews.apache.org/r/2126/
bq.  -----------------------------------------------------------
bq.  
bq.  (Updated 2011-09-30 00:02:16)
bq.  
bq.  
bq.  Review request for hbase, Michael Stack and Andrew Purtell.
bq.  
bq.  
bq.  Summary
bq.  -------
bq.  
bq.  commit fbf82c17be6b3ecca5a981f5270cf93aac26e479
bq.  Author: Jonathan Hsieh <jon@cloudera.com>
bq.  Date:   Wed Sep 28 10:18:11 2011 -0700
bq.  
bq.      HBASE-4377 [hbck] Offline rebuild .META. from fs data only
bq.      
bq.  
bq.  This patch rebuilds a new .META. table by reading all the .regioninfo files in the hbase
main directory.  It depends on the yet to be committed HBASE-4515 (either my verison or Gary's
version), HBASE-4509, and HBASE-4506.  
bq.  
bq.  Some follow on work includes backporting to 0.90, auto-patching true holes, and adding
documentation.
bq.  
bq.  
bq.  This addresses bug HBASE-4377.
bq.      https://issues.apache.org/jira/browse/HBASE-4377
bq.  
bq.  
bq.  Diffs
bq.  -----
bq.  
bq.    src/main/java/org/apache/hadoop/hbase/master/MasterFileSystem.java b9c850d 
bq.    src/main/java/org/apache/hadoop/hbase/util/HBaseFsck.java 8465724 
bq.    src/main/java/org/apache/hadoop/hbase/util/hbck/OfflineMetaRepair.java PRE-CREATION

bq.    src/test/java/org/apache/hadoop/hbase/util/TestHBaseFsck.java fae0881 
bq.    src/test/java/org/apache/hadoop/hbase/util/hbck/HbckTestingUtil.java PRE-CREATION 
bq.    src/test/java/org/apache/hadoop/hbase/util/hbck/TestOfflineMetaRebuild.java PRE-CREATION

bq.  
bq.  Diff: https://reviews.apache.org/r/2126/diff
bq.  
bq.  
bq.  Testing
bq.  -------
bq.  
bq.  An earlier version of this code (backported to 0.90) was used to diagnose and repair
a cluster that had 2700 inconsistencies due to failed splits (the cluster was underprovisioned
memory-wise, and on restart, the some regions would start splitting and then die due to oome's).
 This was not actually used on a live cluster -- it was used to reconstruct a .META. from
.regioninfo's laid out in hbase's directory structure.
bq.  
bq.  Note also that this is not an automatic fix -- whenever any problems are found, this
bails out but dumps info on holes, suggests some fixes, and displays sets of overlapping regions.
 It is up to the user to merge regions, to create .regioninfo files to plug hole, and to do
any potential data loosing operations.
bq.  
bq.  The tests demonstrate current expected behavior -- rebuild meta if things line up, and
fail without making modifications if holes or overlaps exist.
bq.  
bq.  
bq.  Thanks,
bq.  
bq.  jmhsieh
bq.  
bq.


                
> [hbck] Offline rebuild .META. from fs data only.
> ------------------------------------------------
>
>                 Key: HBASE-4377
>                 URL: https://issues.apache.org/jira/browse/HBASE-4377
>             Project: HBase
>          Issue Type: New Feature
>            Reporter: Jonathan Hsieh
>            Assignee: Jonathan Hsieh
>         Attachments: 0001-HBASE-4377-hbck-Offline-rebuild-.META.-from-fs-data-.patch
>
>
> In a worst case situation, it may be helpful to have an offline .META. rebuilder that
just looks at the file system's .regioninfos and rebuilds meta from scratch.  Users could
move bad regions out until there is a clean rebuild.  
> It would likely fill in region split holes.  Follow on work could given options to merge
or select regions that overlap, or do online rebuilds.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message