hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jason Lowe (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HADOOP-14881) LoadGenerator should use Time.monotonicNow() to measure durations
Date Thu, 21 Sep 2017 16:17:00 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-14881?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16175029#comment-16175029

Jason Lowe commented on HADOOP-14881:

Thanks for the patch!

The TestSFTPFileSystem change looks unrelated.

Time.monotonicNow returns a value that only makes sense within the context of a process when
used for deltas of those values to measure elapsed time.  The Javadoc for System.nanoTime,
which Time.montonicNow trivially wraps, states as such:
This method can only be used to measure elapsed time and is not related to any other notion
of system or wall-clock time. The value returned represents nanoseconds since some fixed but
arbitrary origin time (perhaps in the future, so values may be negative). The same origin
is used by all invocations of this method in an instance of a Java virtual machine; other
virtual machine instances are likely to use a different origin.

In LoadGenerator#submitAsMapReduce it stores the monotonicNow value placed into LoadGenerator.startTime
into a conf which will be shipped to another JVM as part of a MapReduce job.  That value likely
will not make sense in the context of the new job.

Similarly, LoadGenerator.startTime can be initialized via a command-line argument.  That argument
will always be specified as seconds since the standard epoch, whereas montonicNow will not
be.  Therefore we cannot change LoadGenerator.startTime to use Time.monotonicNow.  The use
of monotonicNow may be OK within the read, write, list, and genFile methods, but it's not
going to be OK in the LoadGenerator#barrier method.

> LoadGenerator should use Time.monotonicNow() to measure durations
> -----------------------------------------------------------------
>                 Key: HADOOP-14881
>                 URL: https://issues.apache.org/jira/browse/HADOOP-14881
>             Project: Hadoop Common
>          Issue Type: Sub-task
>            Reporter: Chetna Chaudhari
>            Assignee: Bharat Viswanadham
>         Attachments: HADOOP-14881.01.patch

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org

View raw message