hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Eric Zhang <ezh...@yahoo-inc.com>
Subject too many open files error
Date Fri, 26 Sep 2008 22:09:20 GMT
Hi,
I encountered following FileNotFoundException resulting from "too many 
open files" error when i tried to run a job.  The job had been run for 
several times before without problem.  I am confused by the exception 
because my code closes all the files and even it doesn't,  the job only 
have only 10-20 small input/output files.   The limit on the open file 
on my box is 1024.    Besides, the error seemed to happen even before 
the task was executed, I am using 0.17 version.   I'd appreciate if 
somebody can shed some light on this issue.  BTW, the job ran ok after i 
restarted hadoop.    Yes, the hadoop-site.xml did exist in that directory. 

java.lang.RuntimeException: java.io.FileNotFoundException: 
/home/y/conf/hadoop/hadoop-site.xml (Too many open files)
        at 
org.apache.hadoop.conf.Configuration.loadResource(Configuration.java:901)
        at 
org.apache.hadoop.conf.Configuration.loadResources(Configuration.java:804)
        at 
org.apache.hadoop.conf.Configuration.getProps(Configuration.java:772)
        at org.apache.hadoop.conf.Configuration.get(Configuration.java:272)
        at 
org.apache.hadoop.conf.Configuration.getBoolean(Configuration.java:414)
        at 
org.apache.hadoop.mapred.JobConf.getKeepFailedTaskFiles(JobConf.java:306)
        at 
org.apache.hadoop.mapred.TaskTracker$TaskInProgress.setJobConf(TaskTracker.java:1487)
        at 
org.apache.hadoop.mapred.TaskTracker.launchTaskForJob(TaskTracker.java:722)
        at 
org.apache.hadoop.mapred.TaskTracker.localizeJob(TaskTracker.java:716)
        at 
org.apache.hadoop.mapred.TaskTracker.startNewTask(TaskTracker.java:1274)
        at 
org.apache.hadoop.mapred.TaskTracker.offerService(TaskTracker.java:915)
        at org.apache.hadoop.mapred.TaskTracker.run(TaskTracker.java:1310)
        at org.apache.hadoop.mapred.TaskTracker.main(TaskTracker.java:2251)
Caused by: java.io.FileNotFoundException: 
/home/y/conf/hadoop/hadoop-site.xml (Too many open files)
        at java.io.FileInputStream.open(Native Method)
        at java.io.FileInputStream.<init>(FileInputStream.java:106)
        at java.io.FileInputStream.<init>(FileInputStream.java:66)
        at 
sun.net.www.protocol.file.FileURLConnection.connect(FileURLConnection.java:70)
        at 
sun.net.www.protocol.file.FileURLConnection.getInputStream(FileURLConnection.java:161)
        at 
com.sun.org.apache.xerces.internal.impl.XMLEntityManager.setupCurrentEntity(XMLEntityManager.java:653)
        at 
com.sun.org.apache.xerces.internal.impl.XMLVersionDetector.determineDocVersion(XMLVersionDetector.java:186)
        at 
com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:771)
        at 
com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:737)
        at 
com.sun.org.apache.xerces.internal.parsers.XMLParser.parse(XMLParser.java:107)
        at 
com.sun.org.apache.xerces.internal.parsers.DOMParser.parse(DOMParser.java:225)
        at 
com.sun.org.apache.xerces.internal.jaxp.DocumentBuilderImpl.parse(DocumentBuilderImpl.java:283)
        at javax.xml.parsers.DocumentBuilder.parse(DocumentBuilder.java:180)
        at 
org.apache.hadoop.conf.Configuration.loadResource(Configuration.java:832)
        ... 12 more


Sometimes it gave me this message:
java.io.IOException: Cannot run program "bash": java.io.IOException: 
error=24, Too many open files
        at java.lang.ProcessBuilder.start(ProcessBuilder.java:459)
        at org.apache.hadoop.util.Shell.runCommand(Shell.java:149)
        at org.apache.hadoop.util.Shell.run(Shell.java:134)
        at org.apache.hadoop.fs.DF.getAvailable(DF.java:73)
        at 
org.apache.hadoop.fs.LocalDirAllocator$AllocatorPerContext.getLocalPathForWrite(LocalDirAllocator.java:296)
        at 
org.apache.hadoop.fs.LocalDirAllocator.getLocalPathForWrite(LocalDirAllocator.java:124)
        at org.apache.hadoop.mapred.TaskRunner.run(TaskRunner.java:133)
Caused by: java.io.IOException: java.io.IOException: error=24, Too many 
open files
        at java.lang.UNIXProcess.<init>(UNIXProcess.java:148)
        at java.lang.ProcessImpl.start(ProcessImpl.java:65)
        at java.lang.ProcessBuilder.start(ProcessBuilder.java:452)
        ... 6 more



-- 
Eric Zhang
408-349-2466
Vespa Content team


Mime
View raw message