hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Konstantin Shvachko (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-4867) metaSave NPEs when there are invalid blocks in repl queue.
Date Tue, 04 Jun 2013 01:18:20 GMT

    [ https://issues.apache.org/jira/browse/HDFS-4867?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13673904#comment-13673904
] 

Konstantin Shvachko commented on HDFS-4867:
-------------------------------------------

metaSave is probably a casualty here. Should we take a look at why orphaned / missing blocks
are kept in replication queues in the first place?
It seems that when we delete a file blocks can also be removed from replication queue, because
what is the point of replicating them if they don't belong to any files.

It still makes sense to have this case covered in metaSave().
The patch looks good. Couple of nits:
# Could you remove 3 unused imports in the test.
# Also it would be good to close BufferedReader in the end of both test cases.
                
> metaSave NPEs when there are invalid blocks in repl queue.
> ----------------------------------------------------------
>
>                 Key: HDFS-4867
>                 URL: https://issues.apache.org/jira/browse/HDFS-4867
>             Project: Hadoop HDFS
>          Issue Type: Bug
>          Components: namenode
>    Affects Versions: 0.23.7, 2.0.4-alpha
>            Reporter: Kihwal Lee
>            Assignee: Plamen Jeliazkov
>             Fix For: 3.0.0
>
>         Attachments: HDFS-4867.trunk.patch
>
>
> Since metaSave cannot get the inode holding a orphaned/invalid block, it NPEs and stops
generating further report. Normally ReplicationMonitor removes them quickly, but if the queue
is huge, it takes very long time. Also in safe mode, they stay.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message