hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Bwolen Yang" <wbwo...@gmail.com>
Subject Re: write and sort performance
Date Sat, 09 Jun 2007 01:17:42 GMT
> Please try Hadoop 0.13.0.

The multiple random writers case now completes in 278sec (from
294sec).  This diff is not big.  In comparison, the sorter improvement
is more impressive.

For the sorter, now the map phase completes in a tighter range bound
55sec to 79sec (instead of 59sec to 139sec).    This speedup (from
scheduling?) the overall running time significantly (890sec vs
1345sec).   The performance of reduce phase is similar (both shuffle
and sort cases).

Let me summarize my remaining questions in the next email.  Looks like
my original email is way too long to get specific answers.



View raw message