hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "suja s (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-3512) Delay in scanning blocks at DN side when there are huge number of blocks
Date Fri, 08 Jun 2012 14:37:23 GMT

    [ https://issues.apache.org/jira/browse/HDFS-3512?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13291804#comment-13291804

suja s commented on HDFS-3512:

Thank You Amith for taking a look. I have applied the patch and tested with around 6 lakh
blocks in my cluster and scanning is happening as expected.
> Delay in scanning blocks at DN side when there are huge number of blocks
> ------------------------------------------------------------------------
>                 Key: HDFS-3512
>                 URL: https://issues.apache.org/jira/browse/HDFS-3512
>             Project: Hadoop HDFS
>          Issue Type: Bug
>          Components: data-node
>    Affects Versions: 2.0.1-alpha
>            Reporter: suja s
>            Assignee: amith
>         Attachments: HDFS-3512.patch
> Block scanner maintains the full list of blocks at DN side in a map and there is no differentiation
between the blocks which are already scanned and the ones not scanend. For every check (ie
every 5 secs) it will pick one block and scan. There are chances that it chooses a block which
is already scanned which leads to further delay in scanning of blcoks which are yet to be

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira


View raw message