hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Owen O'Malley <o...@yahoo-inc.com>
Subject Re: Out of Memory during Sorts
Date Mon, 12 Jun 2006 14:29:44 GMT

On Jun 11, 2006, at 8:07 AM, Dennis Kubes wrote:

> Can someone lead me in the right direction as to configuring settings 
> for large sorting operations > 1M rows.  I keep getting out of memory 
> exceptions during the sort phase.  Here are my current settings.  I 
> have 2G heap space on each box.

In general, it would help to have stack traces so that we could tell 
what part of the sort was running out of memory. What was your jvm heap 
size set to? It is interesting that increasing the sort factor got you 
around the problem.

-- Owen

View raw message