hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Chris Nauroth (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-316) YARN container launch may exceed maximum Windows command line length due to long classpath
Date Wed, 23 Jan 2013 18:23:14 GMT

    [ https://issues.apache.org/jira/browse/YARN-316?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13560905#comment-13560905

Chris Nauroth commented on YARN-316:

Chris, do you see any reason why the Class-Path manifest approach could not used for all OSes?

I initially tested the Class-Path manifest change on Mac without the {{if (Shell.WINDOWS)}}
guard, so this approach definitely can work across platforms.  Comments in HADOOP-8899, which
was a similar change for branch-1-win MapReduce, give the rationale for limiting the change
to Windows: platforms that don't have the command line length limitation don't need to suffer
the small performance hit of creating the extra jar.

It's a very small performance hit though, and it's a one-time initialization cost, so long-running
jobs will notice it less than short jobs.  If we'd prefer to keep the approach consistent,
then we can do that.  What do you think?

Regarding the extra leading '0' removal in setPermissions, just make sure this leading zero
is not used to force an octal interpretation of the mask.

I think we're OK on this.  The 'o' in the format string (not the length of the zero-padding)
gives us octal formatting of the string prior to calling chmod or winutils.  Neither chmod
nor winutils need a leading zero on the input.  They always interpret the input as an octal
mode.  I ran the full test suite after this change, and it didn't cause any test failures.

> YARN container launch may exceed maximum Windows command line length due to long classpath
> ------------------------------------------------------------------------------------------
>                 Key: YARN-316
>                 URL: https://issues.apache.org/jira/browse/YARN-316
>             Project: Hadoop YARN
>          Issue Type: Bug
>          Components: nodemanager
>    Affects Versions: 3.0.0, trunk-win
>            Reporter: Chris Nauroth
>            Assignee: Chris Nauroth
>         Attachments: YARN-316-branch-trunk-win.1.patch, YARN-316-branch-trunk-win.2.patch
> On Windows, a command line longer than 8192 characters will fail.  This can cause YARN
container launch to fail on Windows if the classpath argument exceeds this limit.

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

View raw message