tez-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Siddharth Seth (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (TEZ-1424) Fixes to DAG text representation in debug mode
Date Tue, 21 Oct 2014 22:21:33 GMT

    [ https://issues.apache.org/jira/browse/TEZ-1424?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14179224#comment-14179224

Siddharth Seth commented on TEZ-1424:

- Javadocs for TEZ_GENERATE_DEBUG_ARTIFACTS - to reflect the text plan being logged rather
than the dag visualization.
bq. Patch uses DAG_ID in the filename for correlating filename with the DAG. If we have to
use DagPlan, we need to provide some unique number in the filename.
Can we log this at the same place wher the dag visualization is generated in that case. That's
before any signal is sent to start the DAG - which would make it less prone to being logeed
after a potential failure.
- In getCredentialsInfo, along with the service - it'd be useful to log the KIND as well.
- TextFormat.printField(entry.getKey(), entry.getValue(), sb); - this handles logging of fields
a little better - with the name of the field getting logged correctly.
Also, credentials checking, instead of comparing the object, can be done using PB artifacts
{code}if (descriptor.getNumber() != DAGProtos.DAGPlan.CREDENTIALS_BINARY_FIELD_NUMBER) {{code}

> Fixes to DAG text representation in debug mode
> ----------------------------------------------
>                 Key: TEZ-1424
>                 URL: https://issues.apache.org/jira/browse/TEZ-1424
>             Project: Apache Tez
>          Issue Type: Bug
>            Reporter: Siddharth Seth
>            Assignee: Rajesh Balamohan
>            Priority: Critical
>         Attachments: TEZ-1424.1.patch, TEZ-1424.2.patch
> Several fixes required
> - Don't log entire tokens, just the identifier should be enough
> - DAG ID (or unique number needs to be used). Otherwise we get only one file per session
> - This should not go into the local-directory - since that isn't accessible. Instead
the log directory would be a better place.
> Marking as critical for 0.5.1 since this is very useful for debugging.

This message was sent by Atlassian JIRA

View raw message