hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Owen O'Malley <omal...@apache.org>
Subject Re: Massive discrepancies in job's bytes written/read
Date Wed, 18 Mar 2009 05:20:19 GMT

On Mar 17, 2009, at 7:44 PM, Bryan Duxbury wrote:

> There is no compression in the mix for us, so that's not the culprit.
> I'd be sort of willing to believe that spilling and sorting play a  
> role in this, but, wow, over 10x read and write? That seems like a  
> big problem.

It happened recently to me too. It was off by 6x. The strange thing  
was that the individual tasks looked right. It was just the aggregate  
that was wrong.

-- Owen

View raw message