hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Colin Patrick McCabe (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HADOOP-12975) Add jitter to CachingGetSpaceUsed's thread
Date Tue, 10 May 2016 20:43:12 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-12975?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15278854#comment-15278854

Colin Patrick McCabe commented on HADOOP-12975:

Thanks, [~eclark].

169	            // add/subtract the jitter.
170	            refreshInterval +=
171	                ThreadLocalRandom.current()
172	                                 .nextLong(jitter, jitter);
Hmm, is this a typo?  It seems like this is always going to return exactly 'jitter' since
the 'least' and the 'bound' arguments are the same?  That seems to defeat the point of randomization.

126	        if (configuration == null) {
127	          return DEFAULT_JITTER;
128	        }
Can we throw an exception in {{GetSpaceUsed#build}} if {{conf == null}}?  It's a weird special
case to have no {{Configuration}} object, and I'm not sure why we'd ever want to do that.
 Then this function could just be {{return this.conf.getLong(JITTER_KEY, DEFAULT_JITTER);}}.

> Add jitter to CachingGetSpaceUsed's thread
> ------------------------------------------
>                 Key: HADOOP-12975
>                 URL: https://issues.apache.org/jira/browse/HADOOP-12975
>             Project: Hadoop Common
>          Issue Type: Sub-task
>    Affects Versions: 2.9.0
>            Reporter: Elliott Clark
>            Assignee: Elliott Clark
>         Attachments: HADOOP-12975v0.patch, HADOOP-12975v1.patch, HADOOP-12975v2.patch,
HADOOP-12975v3.patch, HADOOP-12975v4.patch, HADOOP-12975v5.patch
> Running DU across lots of disks is very expensive and running all of the processes at
the same time creates a noticeable IO spike. We should add some jitter.

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org

View raw message