hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Colin Patrick McCabe (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HADOOP-12975) Add jitter to CachingGetSpaceUsed's thread
Date Tue, 10 May 2016 20:43:12 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-12975?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15278854#comment-15278854
] 

Colin Patrick McCabe commented on HADOOP-12975:
-----------------------------------------------

Thanks, [~eclark].

{code}
169	            // add/subtract the jitter.
170	            refreshInterval +=
171	                ThreadLocalRandom.current()
172	                                 .nextLong(jitter, jitter);
{code}
Hmm, is this a typo?  It seems like this is always going to return exactly 'jitter' since
the 'least' and the 'bound' arguments are the same?  That seems to defeat the point of randomization.
https://docs.oracle.com/javase/7/docs/api/java/util/concurrent/ThreadLocalRandom.html#nextLong(long,%20long)

{code}
126	        if (configuration == null) {
127	          return DEFAULT_JITTER;
128	        }
{code}
Can we throw an exception in {{GetSpaceUsed#build}} if {{conf == null}}?  It's a weird special
case to have no {{Configuration}} object, and I'm not sure why we'd ever want to do that.
 Then this function could just be {{return this.conf.getLong(JITTER_KEY, DEFAULT_JITTER);}}.

> Add jitter to CachingGetSpaceUsed's thread
> ------------------------------------------
>
>                 Key: HADOOP-12975
>                 URL: https://issues.apache.org/jira/browse/HADOOP-12975
>             Project: Hadoop Common
>          Issue Type: Sub-task
>    Affects Versions: 2.9.0
>            Reporter: Elliott Clark
>            Assignee: Elliott Clark
>         Attachments: HADOOP-12975v0.patch, HADOOP-12975v1.patch, HADOOP-12975v2.patch,
HADOOP-12975v3.patch, HADOOP-12975v4.patch, HADOOP-12975v5.patch
>
>
> Running DU across lots of disks is very expensive and running all of the processes at
the same time creates a noticeable IO spike. We should add some jitter.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org


Mime
View raw message