hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Mark Wagner (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (HDFS-4549) WebHDFS hits a Jetty performance issue
Date Thu, 18 Apr 2013 19:34:15 GMT

     [ https://issues.apache.org/jira/browse/HDFS-4549?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel

Mark Wagner updated HDFS-4549:

    Attachment: HDFS-4549.2.patch

I've backported HDFS-3577, HDFS-3318, and HDFS-3788 as best as possible. However, I'm seeing
performance problems above 2GB. It looks like when the MessageBodyWriter reports a length
greater than 2GB, Jersey is reverting to chunked transfer, causing the previously seen performance
problems with Jetty.

I tested this on a 2.0.3 deployment and got the same results (chunked transfer above a certain

[~szetszwo], can you expand on the motivation for using MessageBodyWriter and OpenEntity instead
of manually setting the header values? Would it be problematic to make all branches use the
method of my first patch? It seems that Jersey is taking the MessageBodyWriter information
only as a suggestion.
> WebHDFS hits a Jetty performance issue
> --------------------------------------
>                 Key: HDFS-4549
>                 URL: https://issues.apache.org/jira/browse/HDFS-4549
>             Project: Hadoop HDFS
>          Issue Type: Bug
>          Components: webhdfs
>    Affects Versions: 1.1.2
>            Reporter: Mark Wagner
>            Assignee: Mark Wagner
>         Attachments: example4549.txt, HDFS-4549.1.patch, HDFS-4549.2.patch
> WebHDFS on branch-1 is hitting a Jetty issue for me when it does chunked transfers. This
is the same Jetty issue as MAPREDUCE-4399. I have not observed this on trunk.

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

View raw message