hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Eli Collins (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-3475) Make the replication monitor multipliers configurable
Date Thu, 28 Jun 2012 17:48:44 GMT

    [ https://issues.apache.org/jira/browse/HDFS-3475?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13403288#comment-13403288
] 

Eli Collins commented on HDFS-3475:
-----------------------------------

+1  Looks good.   Style nit going forward: no need to use "this.foo" when referencing members
in the constructor, understand you're doing so to be consistent with the surrounding code
(which doesn't need to do so either)

Per offline conversation Harsh tested this on a real cluster with higher multiplier and verfied
that re-replication proceeded in kind.  Harsh, seems like the current defaults (which you've
preserved here) should be revisited? They seem a little low to me, if you concur please file
a separate jira for that (to raise them).
                
> Make the replication monitor multipliers configurable
> -----------------------------------------------------
>
>                 Key: HDFS-3475
>                 URL: https://issues.apache.org/jira/browse/HDFS-3475
>             Project: Hadoop HDFS
>          Issue Type: Improvement
>    Affects Versions: 2.0.0-alpha
>            Reporter: Harsh J
>            Assignee: Harsh J
>            Priority: Trivial
>         Attachments: HDFS-3475.patch, HDFS-3475.patch, HDFS-3475.patch
>
>
> BlockManager currently hardcodes the following two constants:
> {code}
> private static final int INVALIDATE_WORK_PCT_PER_ITERATION = 32;
> private static final int REPLICATION_WORK_MULTIPLIER_PER_ITERATION = 2;
> {code}
> These are used to throttle/limit the amount of deletion and replication-to-other-DN work
done per heartbeat interval of a live DN.
> Not many have had reasons to want these changed so far but there have been a few requests
I've faced over the past year from a variety of clusters I've helped maintain. I think with
the improvements in disks and network thats already started to be rolled out in production
environments out there, changing these may start making sense to some.
> Lets at least make it advanced-configurable with proper docs that warn adequately, with
the defaults being what they are today. With hardcodes, it comes down to a recompile for admins,
which is not something they may like.
> Please let me know your thoughts.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message