hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Chris Douglas (JIRA)" <j...@apache.org>
Subject [jira] Updated: (HADOOP-1650) Upgrade Jetty to 6.x
Date Fri, 21 Nov 2008 03:20:44 GMT

     [ https://issues.apache.org/jira/browse/HADOOP-1650?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel

Chris Douglas updated HADOOP-1650:

    Attachment: shuffleData.zip

To test performance against Jetty5, I ran a 400 node shuffle test. 40 maps wrote 5GB each,
feeding 400 reducers running on 385 nodes. Jetty6 finished in 684s, Jetty5 in 711; a negligible
difference. In the attached graphs, each of the two runs is visible (two nodes in the cluster
(tt1, tt2) that ran map jobs from both runs). The run on the left is Jetty6, the run on the
right is Jetty5.

|| Filename || Tracking || Notes ||
| \*net | Network | Nearly identical; included to show when the map task ended and when the
shuffle began |
| \*mem | Memory | Neither Jetty5 nor Jetty6 had substantial footprints during the shuffle.
| \*conn | TCP connections | The rate of failed connections in Jetty5, contrasted with Jetty6,
is encouraging |
| \*cpu | CPU | Comparable profiles from both Jetty5 and Jetty6 on both nodes. |
| \*load | Load | Slightly higher load running Jetty5. The spike on tt1 running Jetty5 is
worth mentioning. |

> Upgrade Jetty to 6.x
> --------------------
>                 Key: HADOOP-1650
>                 URL: https://issues.apache.org/jira/browse/HADOOP-1650
>             Project: Hadoop Core
>          Issue Type: Improvement
>          Components: mapred
>            Reporter: Devaraj Das
>            Assignee: Devaraj Das
>         Attachments: 1650-0.patch, 1650-1_core.patch, 1650-1_merge.patch, 1650-1_proxy_delta.patch,
1650-2.patch, 1650-20080120.patch, 1650-jetty6.1.9.patch, 1650-jetty6.1.9.v1.patch, hadoop-1650-jetty6.1.5.patch,
hadoop-jetty6.1.4-lib.tar.gz, hadoop-jetty6.1.6-lib.tar.gz, hadoop-jetty6.1.9-lib.tar.gz,
jetty-hadoop-6.1.6.patch, jetty-hbase.patch, jetty6.1.4.patch, jetty6.1.6.patch, shuffleData.zip
> This is the third attempt at moving to jetty6. Apparently, the jetty-6.1.4 has fixed
some of the issues we discovered in jetty during HADOOP-736 and HADOOP-1273. I'd like to keep
this issue open for sometime so that we have enough time to test out things.

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message