hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Devaraj Das (JIRA)" <j...@apache.org>
Subject [jira] Updated: (HADOOP-816) Allow the sort benchmark to set a buffersize for the map buffer
Date Tue, 12 Dec 2006 11:08:22 GMT
     [ http://issues.apache.org/jira/browse/HADOOP-816?page=all ]

Devaraj Das updated HADOOP-816:

    Status: Patch Available  (was: Open)

> Allow the sort benchmark to set a buffersize for the map buffer
> ---------------------------------------------------------------
>                 Key: HADOOP-816
>                 URL: http://issues.apache.org/jira/browse/HADOOP-816
>             Project: Hadoop
>          Issue Type: Improvement
>          Components: mapred
>            Reporter: Devaraj Das
>         Assigned To: Devaraj Das
>         Attachments: 816.patch
> Discovered that framework merges are the hotspots where most time is spent in the sort
benchmark. With HADOOP-331, the Map phase could potentially do a merge of the spills (this
merge was not done pre-HADOOP-331), and then there is one compulsory merge on each reduce.
It may be good to avoid the merge in the Map phase, if possible.

This message is automatically generated by JIRA.
If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira


View raw message