tez-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hitesh Shah (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (TEZ-345) Command line -D parameters no longer respected in the AM
Date Tue, 06 Aug 2013 20:54:47 GMT

    [ https://issues.apache.org/jira/browse/TEZ-345?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13731250#comment-13731250

Hitesh Shah commented on TEZ-345:

In createFinalAMConf, are we ignoring amConf being passed in as null? Also, as Sid mentioned
earlier, we need a distinction on what configs are across DAGs and only pass those ones in
to the AM.

Missing debug level option to serialize amConf in text mode ( similar option exists for the
dag plan )?

s/DAGPlan.Builder jobBuilder/DAGPlan.Builder dagBuilder/ ?

In "DAGPlan createDag(Configuration dagConf)", can dagConf be null? If null, it seems like
settings in the pb get ignored?

Doing "TezConfiguration conf = new TezConfiguration(new YarnConfiguration());" inside the
AM implies reliance on a tez-site.xml on the cluster which can potentially have different
settings from the client and raising a question on what is the source of truth. Might be preferable
to do "Configuration conf = new YarnConfiguration()" and change the code to just use Configuration
in DAGAppMaster#main.

Follow-up jira for a unit test?

> Command line -D parameters no longer respected in the AM
> --------------------------------------------------------
>                 Key: TEZ-345
>                 URL: https://issues.apache.org/jira/browse/TEZ-345
>             Project: Apache Tez
>          Issue Type: Bug
>            Reporter: Siddharth Seth
>            Assignee: Bikas Saha
>            Priority: Critical
>              Labels: TEZ-0.2.0
>         Attachments: TEZ-345.1.patch
> -D parameters specified at command line are no longer used in the AM (likely broken after
TEZ-323) - specifically scheduler options like heartbeat time, reuse etc. This would have
to be reconsiled with TEZ-340, TEZ-341 which would allow multiple DAGs to be submitted via
a single AM.

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

View raw message