Return-Path: X-Original-To: apmail-tez-commits-archive@minotaur.apache.org Delivered-To: apmail-tez-commits-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id EBDE718B1A for ; Tue, 2 Feb 2016 11:56:52 +0000 (UTC) Received: (qmail 82551 invoked by uid 500); 2 Feb 2016 11:56:52 -0000 Delivered-To: apmail-tez-commits-archive@tez.apache.org Received: (qmail 82516 invoked by uid 500); 2 Feb 2016 11:56:52 -0000 Mailing-List: contact commits-help@tez.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@tez.apache.org Delivered-To: mailing list commits@tez.apache.org Received: (qmail 82507 invoked by uid 99); 2 Feb 2016 11:56:52 -0000 Received: from git1-us-west.apache.org (HELO git1-us-west.apache.org) (140.211.11.23) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 02 Feb 2016 11:56:52 +0000 Received: by git1-us-west.apache.org (ASF Mail Server at git1-us-west.apache.org, from userid 33) id ABF2DDFC8F; Tue, 2 Feb 2016 11:56:52 +0000 (UTC) Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit From: rbalamohan@apache.org To: commits@tez.apache.org Message-Id: <830e54dd7546413c8c2943c7790313cf@git.apache.org> X-Mailer: ASF-Git Admin Mailer Subject: tez git commit: TEZ-3089. TaskConcurrencyAnalyzer can return negative task count with very large jobs (rbalamohan) Date: Tue, 2 Feb 2016 11:56:52 +0000 (UTC) Repository: tez Updated Branches: refs/heads/master 235841f77 -> 89bc6abf6 TEZ-3089. TaskConcurrencyAnalyzer can return negative task count with very large jobs (rbalamohan) Project: http://git-wip-us.apache.org/repos/asf/tez/repo Commit: http://git-wip-us.apache.org/repos/asf/tez/commit/89bc6abf Tree: http://git-wip-us.apache.org/repos/asf/tez/tree/89bc6abf Diff: http://git-wip-us.apache.org/repos/asf/tez/diff/89bc6abf Branch: refs/heads/master Commit: 89bc6abf6c3caddb6224a19b7b47010b7bca4eff Parents: 235841f Author: Rajesh Balamohan Authored: Tue Feb 2 03:56:46 2016 -0800 Committer: Rajesh Balamohan Committed: Tue Feb 2 03:56:46 2016 -0800 ---------------------------------------------------------------------- CHANGES.txt | 1 + .../plugins/TaskConcurrencyAnalyzer.java | 24 ++++++++++++++++++-- 2 files changed, 23 insertions(+), 2 deletions(-) ---------------------------------------------------------------------- http://git-wip-us.apache.org/repos/asf/tez/blob/89bc6abf/CHANGES.txt ---------------------------------------------------------------------- diff --git a/CHANGES.txt b/CHANGES.txt index c4c04e8..b7bb98a 100644 --- a/CHANGES.txt +++ b/CHANGES.txt @@ -6,6 +6,7 @@ Release 0.8.3: Unreleased INCOMPATIBLE CHANGES ALL CHANGES: + TEZ-3089. TaskConcurrencyAnalyzer can return negative task count with very large jobs. TEZ-2307. Possible wrong error message when submitting new dag TEZ-2974. Tez tools: TFileRecordReader in tez-tools should support reading >2 GB tfiles. TEZ-3081. Update tez website for trademarks feedback. http://git-wip-us.apache.org/repos/asf/tez/blob/89bc6abf/tez-tools/analyzers/job-analyzer/src/main/java/org/apache/tez/analyzer/plugins/TaskConcurrencyAnalyzer.java ---------------------------------------------------------------------- diff --git a/tez-tools/analyzers/job-analyzer/src/main/java/org/apache/tez/analyzer/plugins/TaskConcurrencyAnalyzer.java b/tez-tools/analyzers/job-analyzer/src/main/java/org/apache/tez/analyzer/plugins/TaskConcurrencyAnalyzer.java index 070294f..72f3b36 100644 --- a/tez-tools/analyzers/job-analyzer/src/main/java/org/apache/tez/analyzer/plugins/TaskConcurrencyAnalyzer.java +++ b/tez-tools/analyzers/job-analyzer/src/main/java/org/apache/tez/analyzer/plugins/TaskConcurrencyAnalyzer.java @@ -80,8 +80,28 @@ public class TaskConcurrencyAnalyzer extends TezAnalyzerBase implements Analyzer */ TreeMultiset timeInfoSet = TreeMultiset.create(new Comparator() { @Override public int compare(TimeInfo o1, TimeInfo o2) { - return (o1.timestamp < o2.timestamp) ? -1 : - ((o1.timestamp == o2.timestamp) ? 0 : 1); + if (o1.timestamp < o2.timestamp) { + return -1; + } + + if (o1.timestamp > o2.timestamp) { + return 1; + } + + if (o1.timestamp == o2.timestamp) { + //check event type + if (o1.eventType.equals(o2.eventType)) { + return 0; + } + + if (o1.eventType.equals(EventType.START) + && o2.eventType.equals(EventType.FINISH)) { + return -1; + } else { + return 1; + } + } + return 0; } });