hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Walter Su (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-9260) Improve performance and GC friendliness of startup and FBRs
Date Tue, 20 Oct 2015 07:40:27 GMT

    [ https://issues.apache.org/jira/browse/HDFS-9260?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14964719#comment-14964719
] 

Walter Su commented on HDFS-9260:
---------------------------------

-TreeMap- TreeSet

> Improve performance and GC friendliness of startup and FBRs
> -----------------------------------------------------------
>
>                 Key: HDFS-9260
>                 URL: https://issues.apache.org/jira/browse/HDFS-9260
>             Project: Hadoop HDFS
>          Issue Type: Improvement
>          Components: datanode, namenode, performance
>    Affects Versions: 2.7.1
>            Reporter: Staffan Friberg
>            Assignee: Staffan Friberg
>         Attachments: HDFS Block and Replica Management 20151013.pdf, HDFS-7435.001.patch,
HDFS-7435.002.patch, HDFS-7435.003.patch
>
>
> This patch changes the datastructures used for BlockInfos and Replicas to keep them sorted.
This allows faster and more GC friendly handling of full block reports.
> Would like to hear peoples feedback on this change and also some help investigating/understanding
a few outstanding issues if we are interested in moving forward with this.
> There seems to be some timing issues I hit when testing the patch, not sure if it is
a bug in the patch or something else (most likely the earlier)...
> Tests that fail for me:
>    The issues seems to be that the blocks are not on any storage, so no replication can
occur causing the tests to fail in different ways.
>    TestDecomission.testDecommision
>        If I add a little sleep after the cleanup/delete things seem to work
>    TestDFSStripedOutputStreamWithFailure
>        A couple of tests fails in this class.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message