hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "jiraposter@reviews.apache.org (Commented) (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-5719) Enhance hbck to sideline overlapped mega regions
Date Fri, 06 Apr 2012 03:25:19 GMT

    [ https://issues.apache.org/jira/browse/HBASE-5719?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13247994#comment-13247994
] 

jiraposter@reviews.apache.org commented on HBASE-5719:
------------------------------------------------------



bq.  On 2012-04-06 02:26:10, jmhsieh wrote:
bq.  > Nice feature.  Can we call it something other than "breakXxx"?   Maybe "-sidelineLargeOverlap"

Sure.  Will do.


bq.  On 2012-04-06 02:26:10, jmhsieh wrote:
bq.  > src/main/java/org/apache/hadoop/hbase/util/HBaseFsck.java, line 142
bq.  > <https://reviews.apache.org/r/4649/diff/1/?file=100073#file100073line142>
bq.  >
bq.  >     why commented?
bq.  >

It is not used so I commented it out.


bq.  On 2012-04-06 02:26:10, jmhsieh wrote:
bq.  > src/main/java/org/apache/hadoop/hbase/util/HBaseFsck.java, line 157
bq.  > <https://reviews.apache.org/r/4649/diff/1/?file=100073#file100073line157>
bq.  >
bq.  >     why commented?

It is not used so I commented it out.


bq.  On 2012-04-06 02:26:10, jmhsieh wrote:
bq.  > src/main/java/org/apache/hadoop/hbase/util/HBaseFsck.java, line 174
bq.  > <https://reviews.apache.org/r/4649/diff/1/?file=100073#file100073line174>
bq.  >
bq.  >     break?  maybe a different word?

sideline.


bq.  On 2012-04-06 02:26:10, jmhsieh wrote:
bq.  > src/main/java/org/apache/hadoop/hbase/util/HBaseFsck.java, line 880
bq.  > <https://reviews.apache.org/r/4649/diff/1/?file=100073#file100073line880>
bq.  >
bq.  >     instead of nesting, maybe set hfiles to an empty array?  
bq.  >     
bq.  >     if (hfiles==null) {
bq.  >       hfiles = new FileStatus[0];
bq.  >     }
bq.  >     for (FileStatus hfile : hfiles)
bq.  >     ...

That means an extra object. I think the current checking is fine.


bq.  On 2012-04-06 02:26:10, jmhsieh wrote:
bq.  > src/main/java/org/apache/hadoop/hbase/util/HBaseFsck.java, lines 1572-1574
bq.  > <https://reviews.apache.org/r/4649/diff/1/?file=100073#file100073line1572>
bq.  >
bq.  >     Instead of "None/Multiple" just say how many htds?
bq.  >     
bq.  >

All the htds are printed out, so it should be clear enough.


bq.  On 2012-04-06 02:26:10, jmhsieh wrote:
bq.  > src/main/java/org/apache/hadoop/hbase/util/HBaseFsck.java, line 1834
bq.  > <https://reviews.apache.org/r/4649/diff/1/?file=100073#file100073line1834>
bq.  >
bq.  >     Comment about what you are trying to accomplish with this method?

Will do.


bq.  On 2012-04-06 02:26:10, jmhsieh wrote:
bq.  > src/main/java/org/apache/hadoop/hbase/util/HBaseFsck.java, line 1876
bq.  > <https://reviews.apache.org/r/4649/diff/1/?file=100073#file100073line1876>
bq.  >
bq.  >     LOG.debug/trace?

Will change to LOG.debug


bq.  On 2012-04-06 02:26:10, jmhsieh wrote:
bq.  > src/main/java/org/apache/hadoop/hbase/util/HBaseFsck.java, line 1872
bq.  > <https://reviews.apache.org/r/4649/diff/1/?file=100073#file100073line1872>
bq.  >
bq.  >     LOG.debug/trace?
bq.  >

Will change to LOG.debug, good catch.


bq.  On 2012-04-06 02:26:10, jmhsieh wrote:
bq.  > src/main/java/org/apache/hadoop/hbase/util/RegionSplitCalculator.java, lines 167-215
bq.  > <https://reviews.apache.org/r/4649/diff/1/?file=100074#file100074line167>
bq.  >
bq.  >     This is a really important function.  
bq.  >     
bq.  >     Please java doc (maybe an example?) and at least add a unit tests that cover
edge cases here for this.
bq.  >     
bq.  >     What is the definition of a "megaOverlap"?
bq.  >     
bq.  >     What are the properties of the argument and the return values?  
bq.  >     
bq.  >     What is the intuition behind how the regions selected are selected and returned?

Sure, will do.


bq.  On 2012-04-06 02:26:10, jmhsieh wrote:
bq.  > src/main/java/org/apache/hadoop/hbase/util/HBaseFsck.java, lines 2826-2831
bq.  > <https://reviews.apache.org/r/4649/diff/1/?file=100073#file100073line2826>
bq.  >
bq.  >     wrong comment for this method

Will fix it.


bq.  On 2012-04-06 02:26:10, jmhsieh wrote:
bq.  > src/main/java/org/apache/hadoop/hbase/util/HBaseFsck.java, lines 2896-2904
bq.  > <https://reviews.apache.org/r/4649/diff/1/?file=100073#file100073line2896>
bq.  >
bq.  >     Please update usage with new options.

Will do.


bq.  On 2012-04-06 02:26:10, jmhsieh wrote:
bq.  > src/main/java/org/apache/hadoop/hbase/util/HBaseFsck.java, line 1843
bq.  > <https://reviews.apache.org/r/4649/diff/1/?file=100073#file100073line1843>
bq.  >
bq.  >     Is this really a failure?

This should not happen.  Let me remove this checking.


bq.  On 2012-04-06 02:26:10, jmhsieh wrote:
bq.  > src/main/java/org/apache/hadoop/hbase/util/HBaseFsck.java, line 1755
bq.  > <https://reviews.apache.org/r/4649/diff/1/?file=100073#file100073line1755>
bq.  >
bq.  >     We lost info here.

Will add it back.


bq.  On 2012-04-06 02:26:10, jmhsieh wrote:
bq.  > src/main/java/org/apache/hadoop/hbase/util/HBaseFsck.java, lines 1756-1757
bq.  > <https://reviews.apache.org/r/4649/diff/1/?file=100073#file100073line1756>
bq.  >
bq.  >     Pretty important to note that we only "break" groups that exceed the max number
of regions to merge!

Right, I will add some comment.


bq.  On 2012-04-06 02:26:10, jmhsieh wrote:
bq.  > src/main/java/org/apache/hadoop/hbase/util/HBaseFsck.java, line 1858
bq.  > <https://reviews.apache.org/r/4649/diff/1/?file=100073#file100073line1858>
bq.  >
bq.  >     I think we should bail out in this case.

Agree, will fix it.


- Jimmy


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/4649/#review6713
-----------------------------------------------------------


On 2012-04-04 23:52:50, Jimmy Xiang wrote:
bq.  
bq.  -----------------------------------------------------------
bq.  This is an automatically generated e-mail. To reply, visit:
bq.  https://reviews.apache.org/r/4649/
bq.  -----------------------------------------------------------
bq.  
bq.  (Updated 2012-04-04 23:52:50)
bq.  
bq.  
bq.  Review request for hbase and jmhsieh.
bq.  
bq.  
bq.  Summary
bq.  -------
bq.  
bq.  Make it configurable to sideline some regions in big overlapped groups which hbck doesn't
handle currently.
bq.  
bq.  The regions chose to sideline are those which overlap with most other regions.
bq.  
bq.  
bq.  This addresses bug HBASE-5719.
bq.      https://issues.apache.org/jira/browse/HBASE-5719
bq.  
bq.  
bq.  Diffs
bq.  -----
bq.  
bq.    src/main/java/org/apache/hadoop/hbase/util/HBaseFsck.java 54f9b21 
bq.    src/main/java/org/apache/hadoop/hbase/util/RegionSplitCalculator.java 17678dd 
bq.  
bq.  Diff: https://reviews.apache.org/r/4649/diff
bq.  
bq.  
bq.  Testing
bq.  -------
bq.  
bq.  mvn -PlocalTests -Dtest=TestHBaseFsck* clean test
bq.  
bq.  Also tested in real system to fix inconsistencies.
bq.  
bq.  
bq.  Thanks,
bq.  
bq.  Jimmy
bq.  
bq.


                
> Enhance hbck to sideline overlapped mega regions
> ------------------------------------------------
>
>                 Key: HBASE-5719
>                 URL: https://issues.apache.org/jira/browse/HBASE-5719
>             Project: HBase
>          Issue Type: New Feature
>          Components: hbck
>    Affects Versions: 0.94.0, 0.96.0
>            Reporter: Jimmy Xiang
>            Assignee: Jimmy Xiang
>             Fix For: 0.96.0
>
>         Attachments: hbase-5719.patch
>
>
> If there are too many regions in one overlapped group (by default, more than 10), hbck
currently doesn't merge them since it takes time.
> In this case, we can sideline some regions in the group and break the overlapping to
fix the inconsistency.  Later on, sidelined regions can be bulk loaded manually.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message