Mailing-List: contact issues-help@tez.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@tez.apache.org
Date: Fri, 10 Jul 2015 13:07:04 +0000 (UTC)
From: "Rajesh Balamohan (JIRA)" <jira@apache.org>
To: issues@tez.apache.org
Message-ID: <JIRA.12844170.1436533592000.154228.1436533624660@Atlassian.JIRA>
In-Reply-To: <JIRA.12844170.1436533592000@Atlassian.JIRA>
References: <JIRA.12844170.1436533592000@Atlassian.JIRA>
 <JIRA.12844170.1436533592170@arcas>
Subject: [jira] [Created] (TEZ-2612) Critical path analyzer for DAGs
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit

Rajesh Balamohan created TEZ-2612:
-------------------------------------

             Summary: Critical path analyzer for DAGs
                 Key: TEZ-2612
                 URL: https://issues.apache.org/jira/browse/TEZ-2612
             Project: Apache Tez
          Issue Type: Bug
            Reporter: Rajesh Balamohan


This analyzer plugin/tool can be used to understand the important vertices/tasks of interest in large DAG for perf analysis / finding bottlenecks.  It can be used to find out 1.input dependency, 2. failure dependency, 3. scheduling dependency (may be at later stage).

Creating this as a uber ticket. Getting this detail at vertex level might be possible with the existing logs derived from ATS.  For task level analysis, certain more details are required.

1. Timeline details like when fetch/merge/compute/sort etc are not captured now. These details can possibly be added in TaskCompletionEvent
2. Need additional details like the last event that completed processing in the input (for tracing at task level)
3. Add downstream task attempt that caused the higher level task to get rescheduled/restarted. This can be used in terms of understanding in cases where the task failed due to read-error.


--
This message was sent by Atlassian JIRA
(v6.3.4#6332)