hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "amith (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-3512) Delay in scanning blocks at DN side when there are huge number of blocks
Date Wed, 06 Jun 2012 04:35:23 GMT

    [ https://issues.apache.org/jira/browse/HDFS-3512?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13289944#comment-13289944

amith commented on HDFS-3512:

Following is the report from blockscanner UI

Block report for block pool: BP-1421388031-

Total Blocks                 :   5025
Verified in last hour        :   2658
Verified in last day         :   4940
Verified in last week        :   4940
Verified in last four weeks  :   4940
Verified in SCAN_PERIOD      :   4940
Not yet verified             :     85
Verified since restart       :   4940
Scans since restart          :   4940
Scan errors since restart    :      0
Transient scan errors        :      0
Current scan rate limit KBps :   1024
Progress this period         :     98%
Time left in cur period      : 100.00%

> Delay in scanning blocks at DN side when there are huge number of blocks
> ------------------------------------------------------------------------
>                 Key: HDFS-3512
>                 URL: https://issues.apache.org/jira/browse/HDFS-3512
>             Project: Hadoop HDFS
>          Issue Type: Bug
>          Components: data-node
>    Affects Versions: 2.0.1-alpha
>            Reporter: suja s
>            Assignee: amith
> Block scanner maintains the full list of blocks at DN side in a map and there is no differentiation
between the blocks which are already scanned and the ones not scanend. For every check (ie
every 5 secs) it will pick one block and scan. There are chances that it chooses a block which
is already scanned which leads to further delay in scanning of blcoks which are yet to be

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira


View raw message