hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Staffan Friberg (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (HDFS-9260) Improve performance and GC friendliness of startup and FBRs
Date Fri, 16 Oct 2015 17:23:05 GMT

     [ https://issues.apache.org/jira/browse/HDFS-9260?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel

Staffan Friberg updated HDFS-9260:
    Attachment: HDFS Block and Replica Management 20151013.pdf

> Improve performance and GC friendliness of startup and FBRs
> -----------------------------------------------------------
>                 Key: HDFS-9260
>                 URL: https://issues.apache.org/jira/browse/HDFS-9260
>             Project: Hadoop HDFS
>          Issue Type: Improvement
>          Components: datanode, namenode, performance
>    Affects Versions: 2.7.1
>            Reporter: Staffan Friberg
>         Attachments: HDFS Block and Replica Management 20151013.pdf, HDFS-7435.001.patch
> This patch changes the datastructures used for BlockInfos and Replicas to keep them sorted.
This allows faster and more GC friendly handling of full block reports.
> Would like to hear peoples feedback on this change and also some help investigating/understanding
a few outstanding issues if we are interested in moving forward with this.
> There seems to be some timing issues I hit when testing the patch, not sure if it is
a bug in the patch or something else (most likely the earlier)...
> Tests that fail for me:
>    The issues seems to be that the blocks is not on any storage, so no replication can
occurs causing the tests to fail in different ways.
>    TestDecomission.testDecommision
>        If I add a little sleep after the cleanup/delete things seem to work
>    TestDFSStripedOutputStreamWithFailure
>        A couple of tests fails in this class.

This message was sent by Atlassian JIRA

View raw message