hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Tsz Wo (Nicholas), SZE (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-4551) Change WebHDFS buffersize behavior to improve default performance
Date Wed, 06 Mar 2013 22:50:13 GMT

    [ https://issues.apache.org/jira/browse/HDFS-4551?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13595231#comment-13595231
] 

Tsz Wo (Nicholas), SZE commented on HDFS-4551:
----------------------------------------------

Hi Mark,

Currently, WebHDFS on branch-1 first uses user parameter, then the server config value and
then the default value (4096) for buffer size.  WebHDFS on trunk also does the same (i.e.
(1) user parameter, (2) server conf, (3) default 4096.)  For Hftp, it first uses server config
value and then the default value.

Are you sure that changing the buffer size to 4096 helps the performance?  Have you done any
benchmark?


                
> Change WebHDFS buffersize behavior to improve default performance
> -----------------------------------------------------------------
>
>                 Key: HDFS-4551
>                 URL: https://issues.apache.org/jira/browse/HDFS-4551
>             Project: Hadoop HDFS
>          Issue Type: Bug
>          Components: webhdfs
>    Affects Versions: 1.1.2
>            Reporter: Mark Wagner
>            Assignee: Mark Wagner
>         Attachments: HDFS-4551.1.patch
>
>
> Currently on 1.X branch, the buffer size used to copy bytes to network defaults to io.file.buffer.size.
This causes performance problems if that buffersize is large.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message