Return-Path: X-Original-To: apmail-hadoop-common-commits-archive@www.apache.org Delivered-To: apmail-hadoop-common-commits-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 314BA18B61 for ; Sat, 21 Nov 2015 07:56:11 +0000 (UTC) Received: (qmail 24764 invoked by uid 500); 21 Nov 2015 07:56:07 -0000 Delivered-To: apmail-hadoop-common-commits-archive@hadoop.apache.org Received: (qmail 24540 invoked by uid 500); 21 Nov 2015 07:56:07 -0000 Mailing-List: contact common-commits-help@hadoop.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: common-dev@hadoop.apache.org Delivered-To: mailing list common-commits@hadoop.apache.org Received: (qmail 23130 invoked by uid 99); 21 Nov 2015 07:56:06 -0000 Received: from git1-us-west.apache.org (HELO git1-us-west.apache.org) (140.211.11.23) by apache.org (qpsmtpd/0.29) with ESMTP; Sat, 21 Nov 2015 07:56:06 +0000 Received: by git1-us-west.apache.org (ASF Mail Server at git1-us-west.apache.org, from userid 33) id 3B7E6E2F22; Sat, 21 Nov 2015 07:56:06 +0000 (UTC) Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit From: aengineer@apache.org To: common-commits@hadoop.apache.org Date: Sat, 21 Nov 2015 07:56:28 -0000 Message-Id: In-Reply-To: References: X-Mailer: ASF-Git Admin Mailer Subject: [24/50] [abbrv] hadoop git commit: MAPREDUCE-6499. Add elapsed time for retired job in JobHistoryServer WebUI. Contributed by Lin Yiqun. MAPREDUCE-6499. Add elapsed time for retired job in JobHistoryServer WebUI. Contributed by Lin Yiqun. Project: http://git-wip-us.apache.org/repos/asf/hadoop/repo Commit: http://git-wip-us.apache.org/repos/asf/hadoop/commit/a7f776ff Tree: http://git-wip-us.apache.org/repos/asf/hadoop/tree/a7f776ff Diff: http://git-wip-us.apache.org/repos/asf/hadoop/diff/a7f776ff Branch: refs/heads/HDFS-7240 Commit: a7f776ffdfc9f96d3091d99a2a87157b290cbfe0 Parents: 007c6ce Author: Akira Ajisaka Authored: Tue Nov 17 15:16:47 2015 +0900 Committer: Akira Ajisaka Committed: Tue Nov 17 15:16:47 2015 +0900 ---------------------------------------------------------------------- .../org/apache/hadoop/util/StringUtils.java | 35 ++++++++++++++ hadoop-mapreduce-project/CHANGES.txt | 3 ++ .../mapreduce/v2/hs/webapp/HsJobsBlock.java | 48 ++++++++++++++------ 3 files changed, 72 insertions(+), 14 deletions(-) ---------------------------------------------------------------------- http://git-wip-us.apache.org/repos/asf/hadoop/blob/a7f776ff/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/StringUtils.java ---------------------------------------------------------------------- diff --git a/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/StringUtils.java b/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/StringUtils.java index 1107007..1dff0fa 100644 --- a/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/StringUtils.java +++ b/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/StringUtils.java @@ -287,6 +287,41 @@ public class StringUtils { buf.append("sec"); return buf.toString(); } + + /** + * + * Given the time in long milliseconds, returns a String in the sortable + * format Xhrs, Ymins, Zsec. X, Y, and Z are always two-digit. If the time is + * more than 100 hours ,it is displayed as 99hrs, 59mins, 59sec. + * + * @param timeDiff The time difference to format + */ + public static String formatTimeSortable(long timeDiff) { + StringBuilder buf = new StringBuilder(); + long hours = timeDiff / (60 * 60 * 1000); + long rem = (timeDiff % (60 * 60 * 1000)); + long minutes = rem / (60 * 1000); + rem = rem % (60 * 1000); + long seconds = rem / 1000; + + // if hours is more than 99 hours, it will be set a max value format + if (hours > 99) { + hours = 99; + minutes = 59; + seconds = 59; + } + + buf.append(String.format("%02d", hours)); + buf.append("hrs, "); + + buf.append(String.format("%02d", minutes)); + buf.append("mins, "); + + buf.append(String.format("%02d", seconds)); + buf.append("sec"); + return buf.toString(); + } + /** * Formats time in ms and appends difference (finishTime - startTime) * as returned by formatTimeDiff(). http://git-wip-us.apache.org/repos/asf/hadoop/blob/a7f776ff/hadoop-mapreduce-project/CHANGES.txt ---------------------------------------------------------------------- diff --git a/hadoop-mapreduce-project/CHANGES.txt b/hadoop-mapreduce-project/CHANGES.txt index eb3c45e..8408707 100644 --- a/hadoop-mapreduce-project/CHANGES.txt +++ b/hadoop-mapreduce-project/CHANGES.txt @@ -431,6 +431,9 @@ Release 2.8.0 - UNRELEASED MAPREDUCE-5763. Warn message about httpshuffle in NM logs. (Akira AJISAKA via ozawa) + MAPREDUCE-6499. Add elapsed time for retired job in JobHistoryServer WebUI. + (Lin Yiqun via aajisaka) + OPTIMIZATIONS MAPREDUCE-6376. Add avro binary support for jhist files (Ray Chiang via http://git-wip-us.apache.org/repos/asf/hadoop/blob/a7f776ff/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/main/java/org/apache/hadoop/mapreduce/v2/hs/webapp/HsJobsBlock.java ---------------------------------------------------------------------- diff --git a/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/main/java/org/apache/hadoop/mapreduce/v2/hs/webapp/HsJobsBlock.java b/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/main/java/org/apache/hadoop/mapreduce/v2/hs/webapp/HsJobsBlock.java index 1d6f486..28c8eea 100644 --- a/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/main/java/org/apache/hadoop/mapreduce/v2/hs/webapp/HsJobsBlock.java +++ b/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/main/java/org/apache/hadoop/mapreduce/v2/hs/webapp/HsJobsBlock.java @@ -25,6 +25,8 @@ import org.apache.commons.lang.StringEscapeUtils; import org.apache.hadoop.mapreduce.v2.app.AppContext; import org.apache.hadoop.mapreduce.v2.app.job.Job; import org.apache.hadoop.mapreduce.v2.hs.webapp.dao.JobInfo; +import org.apache.hadoop.util.StringUtils; +import org.apache.hadoop.yarn.util.Times; import org.apache.hadoop.yarn.webapp.hamlet.Hamlet; import org.apache.hadoop.yarn.webapp.hamlet.Hamlet.TABLE; import org.apache.hadoop.yarn.webapp.hamlet.Hamlet.TBODY; @@ -66,7 +68,8 @@ public class HsJobsBlock extends HtmlBlock { th("Maps Total"). th("Maps Completed"). th("Reduces Total"). - th("Reduces Completed")._()._(). + th("Reduces Completed"). + th("Elapsed Time")._()._(). tbody(); LOG.info("Getting list of all Jobs."); // Write all the data into a JavaScript array of arrays for JQuery @@ -90,7 +93,10 @@ public class HsJobsBlock extends HtmlBlock { .append(String.valueOf(job.getMapsTotal())).append("\",\"") .append(String.valueOf(job.getMapsCompleted())).append("\",\"") .append(String.valueOf(job.getReducesTotal())).append("\",\"") - .append(String.valueOf(job.getReducesCompleted())).append("\"],\n"); + .append(String.valueOf(job.getReducesCompleted())).append("\",\"") + .append( + StringUtils.formatTimeSortable(Times.elapsed(job.getStartTime(), + job.getFinishTime(), false))).append("\"],\n"); } //Remove the last comma and close off the array of arrays @@ -103,18 +109,32 @@ public class HsJobsBlock extends HtmlBlock { tbody._(). tfoot(). tr(). - th().input("search_init").$type(InputType.text).$name("submit_time").$value("Submit Time")._()._(). - th().input("search_init").$type(InputType.text).$name("start_time").$value("Start Time")._()._(). - th().input("search_init").$type(InputType.text).$name("finish_time").$value("Finish Time")._()._(). - th().input("search_init").$type(InputType.text).$name("start_time").$value("Job ID")._()._(). - th().input("search_init").$type(InputType.text).$name("start_time").$value("Name")._()._(). - th().input("search_init").$type(InputType.text).$name("start_time").$value("User")._()._(). - th().input("search_init").$type(InputType.text).$name("start_time").$value("Queue")._()._(). - th().input("search_init").$type(InputType.text).$name("start_time").$value("State")._()._(). - th().input("search_init").$type(InputType.text).$name("start_time").$value("Maps Total")._()._(). - th().input("search_init").$type(InputType.text).$name("start_time").$value("Maps Completed")._()._(). - th().input("search_init").$type(InputType.text).$name("start_time").$value("Reduces Total")._()._(). - th().input("search_init").$type(InputType.text).$name("start_time").$value("Reduces Completed")._()._(). + th().input("search_init").$type(InputType.text) + .$name("submit_time").$value("Submit Time")._()._(). + th().input("search_init").$type(InputType.text) + .$name("start_time").$value("Start Time")._()._(). + th().input("search_init").$type(InputType.text) + .$name("finish_time").$value("Finish Time")._()._(). + th().input("search_init").$type(InputType.text) + .$name("job_id").$value("Job ID")._()._(). + th().input("search_init").$type(InputType.text) + .$name("name").$value("Name")._()._(). + th().input("search_init").$type(InputType.text) + .$name("user").$value("User")._()._(). + th().input("search_init").$type(InputType.text) + .$name("queue").$value("Queue")._()._(). + th().input("search_init").$type(InputType.text) + .$name("state").$value("State")._()._(). + th().input("search_init").$type(InputType.text) + .$name("maps_total").$value("Maps Total")._()._(). + th().input("search_init").$type(InputType.text). + $name("maps_completed").$value("Maps Completed")._()._(). + th().input("search_init").$type(InputType.text). + $name("reduces_total").$value("Reduces Total")._()._(). + th().input("search_init").$type(InputType.text). + $name("reduces_completed").$value("Reduces Completed")._()._(). + th().input("search_init").$type(InputType.text). + $name("elapsed_time").$value("Elapsed Time")._()._(). _(). _(). _();