hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hudson (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-5712) Parallelize load of .regioninfo files in diagnostic/repair portion of hbck.
Date Sat, 05 May 2012 02:29:51 GMT

    [ https://issues.apache.org/jira/browse/HBASE-5712?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13268875#comment-13268875
] 

Hudson commented on HBASE-5712:
-------------------------------

Integrated in HBase-0.92-security #106 (See [https://builds.apache.org/job/HBase-0.92-security/106/])
    HBASE-5712 Parallelize load of .regioninfo files in diagnostic/repair portion of hbck
(Revision 1332070)

     Result = SUCCESS
jmhsieh : 
Files : 
* /hbase/branches/0.92/CHANGES.txt
* /hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/util/HBaseFsck.java
* /hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/util/TestHBaseFsck.java

                
> Parallelize load of .regioninfo files in diagnostic/repair portion of hbck.
> ---------------------------------------------------------------------------
>
>                 Key: HBASE-5712
>                 URL: https://issues.apache.org/jira/browse/HBASE-5712
>             Project: HBase
>          Issue Type: Improvement
>          Components: hbck
>    Affects Versions: 0.90.7, 0.92.2, 0.94.0, 0.96.0
>            Reporter: Jonathan Hsieh
>            Assignee: Jonathan Hsieh
>             Fix For: 0.90.7, 0.92.2, 0.94.0, 0.96.0
>
>         Attachments: hbase-5712-90-v2.patch, hbase-5712-90.patch, hbase-5712-v2.patch,
hbase-5712.patch
>
>
> On heavily loaded hdfs's some dfs nodes may not respond quickly and backs off for 60s
before attempting to read data from another datanode.  Portions of the information gathered
from hdfs (.regioninfo files) are loaded serially.  With HBase with clusters with 100's, or
1000's, or 10000's regions encountering these 60s delay blocks progress and can be very painful.
 
> There is already some parallelization of portions of the hdfs information load operations
and the goal here is move the reading of .regioninfos into the parallelized sections..

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message