hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Owen O'Malley (JIRA)" <j...@apache.org>
Subject [jira] Updated: (HADOOP-254) use http to shuffle data between the maps and the reduces
Date Thu, 25 May 2006 16:59:30 GMT
     [ http://issues.apache.org/jira/browse/HADOOP-254?page=all ]

Owen O'Malley updated HADOOP-254:
---------------------------------

    Attachment: http-shuffle.patch

This patch introduces the use of http to shuffle the data from maps to reduces. Using this
patch, the shuffle mostly keeps up with the maps during my sort benchmark.

This patch also allows the task tracker's http servers to try multiple ports until it finds
one that works.

> use http to shuffle data between the maps and the reduces
> ---------------------------------------------------------
>
>          Key: HADOOP-254
>          URL: http://issues.apache.org/jira/browse/HADOOP-254
>      Project: Hadoop
>         Type: Improvement

>   Components: mapred
>     Versions: 0.2.1
>     Reporter: Owen O'Malley
>     Assignee: Owen O'Malley
>      Fix For: 0.3
>  Attachments: http-shuffle.patch
>
> To speed up the shuffle time, I'll use http (via the task tracker's jetty server) to
send the map outputs.

-- 
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
   http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see:
   http://www.atlassian.com/software/jira


Mime
View raw message