Mailing-List: contact mapreduce-issues-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Date: Sat, 26 Aug 2017 00:52:00 +0000 (UTC)
From: "Konstantin Shvachko (JIRA)" <jira@apache.org>
To: mapreduce-issues@hadoop.apache.org
Message-ID: <JIRA.13092042.1501720419000.126783.1503708720193@Atlassian.JIRA>
In-Reply-To: <JIRA.13092042.1501720419000@Atlassian.JIRA>
References: <JIRA.13092042.1501720419000@Atlassian.JIRA> <JIRA.13092042.1501720419976@jira-lw-us.apache.org>
Subject: [jira] [Commented] (MAPREDUCE-6931) Remove TestDFSIO "Total
 Throughput" calculation
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit
archived-at: Sat, 26 Aug 2017 00:52:05 -0000


    [ https://issues.apache.org/jira/browse/MAPREDUCE-6931?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16142506#comment-16142506 ] 

Konstantin Shvachko commented on MAPREDUCE-6931:
------------------------------------------------

Looks like Jenkins is applying full pull request. [~dennishuo] could you please revert last commit, which refactors variable names.

> Remove TestDFSIO "Total Throughput" calculation
> -----------------------------------------------
>
>                 Key: MAPREDUCE-6931
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-6931
>             Project: Hadoop Map/Reduce
>          Issue Type: Bug
>          Components: benchmarks, test
>    Affects Versions: 2.8.0
>            Reporter: Dennis Huo
>            Priority: Trivial
>         Attachments: MAPREDUCE-6931-001.patch
>
>
> The new "Total Throughput" line added in https://issues.apache.org/jira/browse/HDFS-9153 is currently calculated as {{toMB(size) / ((float)execTime)}} and claims to be in units of "MB/s", but {{execTime}} is in milliseconds; thus, the reported number is 1/1000x the actual value:
> {code:java}
>     String resultLines[] = {
>         "----- TestDFSIO ----- : " + testType,
>         "            Date & time: " + new Date(System.currentTimeMillis()),
>         "        Number of files: " + tasks,
>         " Total MBytes processed: " + df.format(toMB(size)),
>         "      Throughput mb/sec: " + df.format(size * 1000.0 / (time * MEGA)),
>         "Total Throughput mb/sec: " + df.format(toMB(size) / ((float)execTime)),
>         " Average IO rate mb/sec: " + df.format(med),
>         "  IO rate std deviation: " + df.format(stdDev),
>         "     Test exec time sec: " + df.format((float)execTime / 1000),
>         "" };
> {code}
> The different calculated fields can also use toMB and a shared milliseconds-to-seconds conversion to make it easier to keep units consistent.


--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

---------------------------------------------------------------------
To unsubscribe, e-mail: mapreduce-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-help@hadoop.apache.org