hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ted Yu <yuzhih...@gmail.com>
Subject Re: Killed : GC overhead limit exceeded
Date Sat, 17 Jul 2010 05:28:27 GMT
Have you tried increasing memory beyond 1GB for your map task ?

I think you have noticed that both OOME came from Pattern.compile().

Please take a look at

I would suggest pre-compiling the three patterns when setting up your mapper
- basically write your own split() and replaceAll().

I recently did something similar. You can find out the performance
improvement by customization -


On Fri, Jul 16, 2010 at 6:06 AM, Some Body <somebody@squareplanet.de> wrote:

> Guess attachments are stripped.
> Here's the memory graph:   http://tinyurl.com/37g3hmu
> Here's the VM Summary:   http://tinyurl.com/36wqzjq
> Alan

  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message