hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From 심탁길 <1004...@nhncorp.com>
Subject Jobtracker is out of memory with 100,000 dummy map tasks job
Date Fri, 26 Sep 2008 06:26:37 GMT
Hi all
Recently I tried 100,000 dummy map tasks job on the 100ea node(2GB, Dual Core, 64Bit machine,
Version: 0.16.4) cluster
Map task does nothing but sleeping one minute
I found that Jobtracker(1GB Heap) consumes about 650MB of heap memory when the job is 50%
After all, the job failed at the 90% of progress because Jobtracker hanged up(?) due to out
of memory.
how do you handle this kind of issue? 
another related issue:
while the above job was being processed, I clicked on the "Pending" on jobdatails.jsp of web
then, Jobtracker consumed 100% of CPU. and 100% CPU status lasted a couple of minutes

  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message