hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hudson (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-7503) Namenode restart after large deletions can cause slow processReport (due to logging)
Date Thu, 11 Dec 2014 14:32:15 GMT

    [ https://issues.apache.org/jira/browse/HDFS-7503?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14242585#comment-14242585
] 

Hudson commented on HDFS-7503:
------------------------------

FAILURE: Integrated in Hadoop-Hdfs-trunk #1969 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk/1969/])
HDFS-7503. Namenode restart after large deletions can cause slow processReport (Arpit Agarwal)
(arp: rev 390642acf35f3d599271617d30ba26c2f6406fc1)
* hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
* hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockManager.java


> Namenode restart after large deletions can cause slow processReport (due to logging)
> ------------------------------------------------------------------------------------
>
>                 Key: HDFS-7503
>                 URL: https://issues.apache.org/jira/browse/HDFS-7503
>             Project: Hadoop HDFS
>          Issue Type: Bug
>          Components: namenode
>    Affects Versions: 1.2.1, 2.6.0
>            Reporter: Arpit Agarwal
>            Assignee: Arpit Agarwal
>         Attachments: HDFS-7503.branch-1.02.patch, HDFS-7503.branch-1.patch, HDFS-7503.trunk.01.patch,
HDFS-7503.trunk.02.patch
>
>
> If a large directory is deleted and namenode is immediately restarted, there are a lot
of blocks that do not belong to any file. This results in a log:
> {code}
> 2014-11-08 03:11:45,584 INFO BlockStateChange (BlockManager.java:processReport(1901))
- BLOCK* processReport: blk_1074250282_509532 on 172.31.44.17:1019 size 6 does not belong
to any file.
> {code}
> This log is printed within FSNamsystem lock. This can cause namenode to take long time
in coming out of safemode.
> One solution is to downgrade the logging level.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message