hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Alejandro Abdelnur (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HADOOP-8899) Classpath exceeds maximum OS limit
Date Tue, 09 Oct 2012 16:14:03 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-8899?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13472503#comment-13472503
] 

Alejandro Abdelnur commented on HADOOP-8899:
--------------------------------------------

You don't want to unjar and jar, as you'll lose in the process any JDK services definitions
in the JARs. I'd suggest the following alternate approach: all JARs in the DistributedCache
classpath should be symlink-ed to a distributedcache-lib/ dir in the current directory and
then 'distributedcached-lib/*' should be added to the classpath.
                
> Classpath exceeds maximum OS limit
> ----------------------------------
>
>                 Key: HADOOP-8899
>                 URL: https://issues.apache.org/jira/browse/HADOOP-8899
>             Project: Hadoop Common
>          Issue Type: Bug
>    Affects Versions: 1-win
>            Reporter: Ahmed El Baz
>
> We hit errors in TT due to long classpath value. One example is Oozie trying to start
a hive job, and it fails with the following error:
>  java.io.IOException: Command exceeds the OS command length limit: 8192, command: "set
CLASSPATH="….
> The classpath includes Hadoop Core + Hive Jars which are in the distributed cache. This
is causing the classpath to be too long giving the error above. A viable long term fix is
to generate a temporary JAR file in the task cache directory which includes all elements in
the long classpath, and then provide this as the "-classpath" argument for the JVM to be spawned.
This is gated for Windows only.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message