hadoop-hdfs-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Staffan Friberg (JIRA)" <j...@apache.org>
Subject [jira] [Created] (HDFS-9260) Improve performance and GC friendliness of startup and FBRs
Date Fri, 16 Oct 2015 17:18:06 GMT
Staffan Friberg created HDFS-9260:
-------------------------------------

             Summary: Improve performance and GC friendliness of startup and FBRs
                 Key: HDFS-9260
                 URL: https://issues.apache.org/jira/browse/HDFS-9260
             Project: Hadoop HDFS
          Issue Type: Improvement
          Components: datanode, namenode, performance
    Affects Versions: 2.7.1
            Reporter: Staffan Friberg


This patch changes the datastructures used for BlockInfos and Replicas to keep them sorted.
This allows faster and more GC friendly handling of full block reports.

Would like to hear peoples feedback on this change and also some help investigating/understanding
a few outstanding issues if we are interested in moving forward with this.

There seems to be some timing issues I hit when testing the patch, not sure if it is a bug
in the patch or something else (most likely the earlier)...

Tests that fail for me:

   The issues seems to be that the blocks is not on any storage, so no replication can occurs
causing the tests to fail in different ways.

   TestDecomission.testDecommision
       If I add a little sleep after the cleanup/delete things seem to work
   TestDFSStripedOutputStreamWithFailure
       A couple of tests fails in this class.





--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message