hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Philippe Gassmann <philippe.gassm...@anyware-tech.com>
Subject Re: OutOfMemoryError in the job tracker
Date Fri, 02 Feb 2007 08:44:23 GMT


Koji Noguchi a écrit :
> Hi Thomas,
> 
> I think we had the same problem before.  Dhruba pointed out,
> 
> http://bugs.sun.com/bugdatabase/view_bug.do;:YfiG?bug_id=4705373
> 

I doubt this is the cause of the problem since our jobtracker JVM is 
started with -Xmx1000m and our jars never exceed 25m...

This issue looks like a huge memory leak since everything is ok for a 
certain amount of time, and after a given time the OutOfMemory exception 
is thrown.

>  > Did someone have a solution to avoid this ?
>  >
> Couple of options.
> 1) Ask the users to submit smaller jar file.
> 2) Use JDK 1.6
> 3) Use 64-bit JVM
> 
> Hope it works.
> 
> Koji
> 
> 
> Thomas Friol wrote:
>> Hi,
>>
>> I am using the 0.10.1 version of hadoop and after less than 24 hour of
>> running, the jobtracker send the following error when it tries to run a
>> job :
>>
>> 2007-02-01 04:10:09,285 ERROR org.apache.hadoop.mapred.JobTracker: Job
>> initialization failed:
>> java.lang.OutOfMemoryError
>>     at java.util.zip.ZipFile.open(Native Method)
>>     at java.util.zip.ZipFile.<init>(ZipFile.java:203)
>>     at java.util.jar.JarFile.<init>(JarFile.java:132)
>>     at java.util.jar.JarFile.<init>(JarFile.java:70)
>>     at sun.misc.URLClassPath$JarLoader.getJarFile(URLClassPath.java:579)
>>     at sun.misc.URLClassPath$JarLoader.<init>(URLClassPath.java:546)
>>     at sun.misc.URLClassPath$3.run(URLClassPath.java:324)
>>     at java.security.AccessController.doPrivileged(Native Method)
>>     at sun.misc.URLClassPath.getLoader(URLClassPath.java:313)
>>     at sun.misc.URLClassPath.getLoader(URLClassPath.java:290)
>>     at sun.misc.URLClassPath.getResource(URLClassPath.java:160)
>>     at java.net.URLClassLoader$1.run(URLClassLoader.java:192)
>>     at java.security.AccessController.doPrivileged(Native Method)
>>     at java.net.URLClassLoader.findClass(URLClassLoader.java:188)
>>     at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
>>     at java.lang.ClassLoader.loadClass(ClassLoader.java:251)
>>     at java.lang.ClassLoader.loadClassInternal(ClassLoader.java:319)
>>     at java.lang.Class.forName0(Native Method)
>>     at java.lang.Class.forName(Class.java:242)
>>     at
>> org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:315) 
>>
>>     at 
>> org.apache.hadoop.conf.Configuration.getClass(Configuration.java:326)
>>     at 
>> org.apache.hadoop.conf.Configuration.getClass(Configuration.java:339)
>>     at org.apache.hadoop.mapred.JobConf.getInputFormat(JobConf.java:263)
>>     at
>> org.apache.hadoop.mapred.JobInProgress.initTasks(JobInProgress.java:123)
>>     at
>> org.apache.hadoop.mapred.JobTracker$JobInitThread.run(JobTracker.java:360) 
>>
>>     at java.lang.Thread.run(Thread.java:595)
>>
>> Did soemone notice the same kind of behavior ?
>> Did someone have a solution to avoid this ?
>>
>> I just found that restarting the jobtracker solves the problem but
>> that's it.
>>
>> Thanks for any help.
>>
> 

Mime
View raw message