hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Tsz Wo (Nicholas), SZE (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-941) Datanode xceiver protocol should allow reuse of a connection
Date Fri, 17 Jun 2011 17:44:47 GMT

    [ https://issues.apache.org/jira/browse/HDFS-941?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13051216#comment-13051216
] 

Tsz Wo (Nicholas), SZE commented on HDFS-941:
---------------------------------------------

Hi Konstantin, don't you agree that the result number of "Throughput" and "Average IO rate"
do not make much sense?  At least the definitions of these two numbers are not clear.  Recall
that we got >1 GB/sec in the past.

{noformat}
10/08/07 00:19:55 INFO fs.TestDFSIO: ----- TestDFSIO ----- : read
10/08/07 00:19:55 INFO fs.TestDFSIO:            Date & time: Sat Aug 07 00:19:55 UTC 2010
10/08/07 00:19:55 INFO fs.TestDFSIO:        Number of files: 2
10/08/07 00:19:55 INFO fs.TestDFSIO: Total MBytes processed: 2048
10/08/07 00:19:55 INFO fs.TestDFSIO:      Throughput mb/sec: 1096.3597430406853
10/08/07 00:19:55 INFO fs.TestDFSIO: Average IO rate mb/sec: 1143.6881103515625
10/08/07 00:19:55 INFO fs.TestDFSIO:  IO rate std deviation: 232.655606509863
10/08/07 00:19:55 INFO fs.TestDFSIO:     Test exec time sec: 28.354
{noformat}

> Datanode xceiver protocol should allow reuse of a connection
> ------------------------------------------------------------
>
>                 Key: HDFS-941
>                 URL: https://issues.apache.org/jira/browse/HDFS-941
>             Project: Hadoop HDFS
>          Issue Type: Improvement
>          Components: data-node, hdfs client
>    Affects Versions: 0.22.0
>            Reporter: Todd Lipcon
>            Assignee: bc Wong
>             Fix For: 0.22.0
>
>         Attachments: 941.22.txt, 941.22.txt, 941.22.v2.txt, 941.22.v3.txt, HDFS-941-1.patch,
HDFS-941-2.patch, HDFS-941-3.patch, HDFS-941-3.patch, HDFS-941-4.patch, HDFS-941-5.patch,
HDFS-941-6.22.patch, HDFS-941-6.patch, HDFS-941-6.patch, HDFS-941-6.patch, fix-close-delta.txt,
hdfs-941.txt, hdfs-941.txt, hdfs-941.txt, hdfs-941.txt, hdfs941-1.png
>
>
> Right now each connection into the datanode xceiver only processes one operation.
> In the case that an operation leaves the stream in a well-defined state (eg a client
reads to the end of a block successfully) the same connection could be reused for a second
operation. This should improve random read performance significantly.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message