hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Eric Payne (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-1257) Race condition on FSNamesystem#recentInvalidateSets introduced by HADOOP-5124
Date Thu, 11 Aug 2011 16:57:27 GMT

    [ https://issues.apache.org/jira/browse/HDFS-1257?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13083226#comment-13083226
] 

Eric Payne commented on HDFS-1257:
----------------------------------

The test errors fall into 2 categoris:
1) Failing in exactly the same way for me either when I build trunk or when I build with the
1257 patch.
2) Not failing for me in trunk.

There are no failures that fail in the patch build that did not also fail in the trunk build.

----
1) Failing in trunk 'ant test' build as well as in patch build:
    org.apache.hadoop.hdfs.server.blockmanagement.TestReplicationPolicy
    org.apache.hadoop.hdfs.server.namenode.TestCheckpoint
        Failing for both on testSeparateEditsDirLocking()
    org.apache.hadoop.hdfs.server.namenode.TestNNThroughputBenchmark
        Failing for both on testNNThroughput()
    org.apache.hadoop.hdfs.server.namenode.TestValidateConfigurationSettings
        Failing for both on testThatMatchingRPCandHttpPortsThrowException()
        Failing for both on testThatDifferentRPCandHttpPortsAreOK()
    org.apache.hadoop.hdfs.TestHDFSServerPorts
        Failing for both on testSecondaryNodePorts()

----
2) Not failing with patched code when I run 'ant test':
    org.apache.hadoop.hdfs.server.datanode.TestBlockRecovery
    org.apache.hadoop.hdfs.server.datanode.TestDataDirs
    org.apache.hadoop.hdfs.server.namenode.TestGetImageServlet
    org.apache.hadoop.hdfs.server.namenode.TestINodeFile
    org.apache.hadoop.hdfs.server.namenode.TestNNLeaseRecovery


> Race condition on FSNamesystem#recentInvalidateSets introduced by HADOOP-5124
> -----------------------------------------------------------------------------
>
>                 Key: HDFS-1257
>                 URL: https://issues.apache.org/jira/browse/HDFS-1257
>             Project: Hadoop HDFS
>          Issue Type: Bug
>          Components: name-node
>    Affects Versions: 0.23.0
>            Reporter: Ramkumar Vadali
>            Assignee: Eric Payne
>             Fix For: 0.23.0
>
>         Attachments: HDFS-1257.1.20110810.patch, HDFS-1257.patch
>
>
> HADOOP-5124 provided some improvements to FSNamesystem#recentInvalidateSets. But it introduced
unprotected access to the data structure recentInvalidateSets. Specifically, FSNamesystem.computeInvalidateWork
accesses recentInvalidateSets without read-lock protection. If there is concurrent activity
(like reducing replication on a file) that adds to recentInvalidateSets, the name-node crashes
with a ConcurrentModificationException.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message