hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Chris Nauroth (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-316) YARN container launch may exceed maximum Windows command line length due to long classpath
Date Wed, 23 Jan 2013 18:23:14 GMT

    [ https://issues.apache.org/jira/browse/YARN-316?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13560905#comment-13560905
] 

Chris Nauroth commented on YARN-316:
------------------------------------

{quote}
Chris, do you see any reason why the Class-Path manifest approach could not used for all OSes?
{quote}

I initially tested the Class-Path manifest change on Mac without the {{if (Shell.WINDOWS)}}
guard, so this approach definitely can work across platforms.  Comments in HADOOP-8899, which
was a similar change for branch-1-win MapReduce, give the rationale for limiting the change
to Windows: platforms that don't have the command line length limitation don't need to suffer
the small performance hit of creating the extra jar.

It's a very small performance hit though, and it's a one-time initialization cost, so long-running
jobs will notice it less than short jobs.  If we'd prefer to keep the approach consistent,
then we can do that.  What do you think?

{quote}
Regarding the extra leading '0' removal in setPermissions, just make sure this leading zero
is not used to force an octal interpretation of the mask.
{quote}

I think we're OK on this.  The 'o' in the format string (not the length of the zero-padding)
gives us octal formatting of the string prior to calling chmod or winutils.  Neither chmod
nor winutils need a leading zero on the input.  They always interpret the input as an octal
mode.  I ran the full test suite after this change, and it didn't cause any test failures.

                
> YARN container launch may exceed maximum Windows command line length due to long classpath
> ------------------------------------------------------------------------------------------
>
>                 Key: YARN-316
>                 URL: https://issues.apache.org/jira/browse/YARN-316
>             Project: Hadoop YARN
>          Issue Type: Bug
>          Components: nodemanager
>    Affects Versions: 3.0.0, trunk-win
>            Reporter: Chris Nauroth
>            Assignee: Chris Nauroth
>         Attachments: YARN-316-branch-trunk-win.1.patch, YARN-316-branch-trunk-win.2.patch
>
>
> On Windows, a command line longer than 8192 characters will fail.  This can cause YARN
container launch to fail on Windows if the classpath argument exceeds this limit.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message