hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Junping Du (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (MAPREDUCE-6931) Remove TestDFSIO "Total Throughput" calculation
Date Fri, 01 Sep 2017 19:19:00 GMT

    [ https://issues.apache.org/jira/browse/MAPREDUCE-6931?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16151042#comment-16151042
] 

Junping Du commented on MAPREDUCE-6931:
---------------------------------------

This JIRA is marked as trivial, but we are in 2.8.2 RC stage. In my practice (different RM
may have different practices), commits under major priority should be skipped at this stage
with balance between importance of fixes and risky of careless code/merge.

> Remove TestDFSIO "Total Throughput" calculation
> -----------------------------------------------
>
>                 Key: MAPREDUCE-6931
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-6931
>             Project: Hadoop Map/Reduce
>          Issue Type: Bug
>          Components: benchmarks, test
>    Affects Versions: 2.8.0
>            Reporter: Dennis Huo
>            Assignee: Dennis Huo
>            Priority: Trivial
>             Fix For: 2.9.0, 3.0.0-beta1, 2.7.5, 2.8.3
>
>         Attachments: MAPREDUCE-6931-001.patch
>
>
> The new "Total Throughput" line added in https://issues.apache.org/jira/browse/HDFS-9153
is currently calculated as {{toMB(size) / ((float)execTime)}} and claims to be in units of
"MB/s", but {{execTime}} is in milliseconds; thus, the reported number is 1/1000x the actual
value:
> {code:java}
>     String resultLines[] = {
>         "----- TestDFSIO ----- : " + testType,
>         "            Date & time: " + new Date(System.currentTimeMillis()),
>         "        Number of files: " + tasks,
>         " Total MBytes processed: " + df.format(toMB(size)),
>         "      Throughput mb/sec: " + df.format(size * 1000.0 / (time * MEGA)),
>         "Total Throughput mb/sec: " + df.format(toMB(size) / ((float)execTime)),
>         " Average IO rate mb/sec: " + df.format(med),
>         "  IO rate std deviation: " + df.format(stdDev),
>         "     Test exec time sec: " + df.format((float)execTime / 1000),
>         "" };
> {code}
> The different calculated fields can also use toMB and a shared milliseconds-to-seconds
conversion to make it easier to keep units consistent.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

---------------------------------------------------------------------
To unsubscribe, e-mail: mapreduce-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-help@hadoop.apache.org


Mime
View raw message