hadoop-hdfs-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Andrew Wang (JIRA)" <j...@apache.org>
Subject [jira] [Created] (HDFS-11023) I/O based throttling of DN replication work
Date Mon, 17 Oct 2016 22:19:59 GMT
Andrew Wang created HDFS-11023:

             Summary: I/O based throttling of DN replication work
                 Key: HDFS-11023
                 URL: https://issues.apache.org/jira/browse/HDFS-11023
             Project: Hadoop HDFS
          Issue Type: Sub-task
          Components: erasure-coding
    Affects Versions: 3.0.0-alpha1
            Reporter: Andrew Wang

Datanode recovery work is currently throttled based on the number of blocks to be recovered.
This is fine in a replicated world, since each block is equally expensive to recover. However,
EC blocks are much more expensive to recover, and the amount depends on the EC policy.

It'd be better to have recovery throttles that accounted for the amount of I/O rather than
just the # of blocks.

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: hdfs-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-help@hadoop.apache.org

View raw message