hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ahmed El Baz (JIRA)" <j...@apache.org>
Subject [jira] [Created] (HADOOP-8899) Classpath exceeds maximum OS limit
Date Mon, 08 Oct 2012 20:46:02 GMT
Ahmed El Baz created HADOOP-8899:

             Summary: Classpath exceeds maximum OS limit
                 Key: HADOOP-8899
                 URL: https://issues.apache.org/jira/browse/HADOOP-8899
             Project: Hadoop Common
          Issue Type: Bug
    Affects Versions: 1-win
            Reporter: Ahmed El Baz

We hit errors in TT due to long classpath value. One example is Oozie trying to start a hive
job, and it fails with the following error:
 java.io.IOException: Command exceeds the OS command length limit: 8192, command: "set CLASSPATH="….

The classpath includes Hadoop Core + Hive Jars which are in the distributed cache. This is
causing the classpath to be too long giving the error above. A viable long term fix is to
generate a temporary JAR file in the task cache directory which includes all elements in the
long classpath, and then provide this as the "-classpath" argument for the JVM to be spawned.
This is gated for Windows only.

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

View raw message