hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Todd Lipcon (Commented) (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-2129) Simplify BlockReader to not inherit from FSInputChecker
Date Wed, 02 Nov 2011 22:49:32 GMT

    [ https://issues.apache.org/jira/browse/HDFS-2129?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13142648#comment-13142648
] 

Todd Lipcon commented on HDFS-2129:
-----------------------------------

On the v2 patch on HDFS-2533, the numbers improve to:

| Threads | Trunk  | HDFS-2533v2           | HDFS-2533 + HDFS-2129 (nonative)  | HDFS-2533
+ HDFS-2129 + native |
| 4 | 226556 KB/s  | 237805 KB/sec (1.05x) |  252639 KB/sec (1.12x)            | 298635 KB/sec
(1.31x) |
| 16 | 377474 KB/s | 499399 KB/sec (1.32x) |  487537 KB/sec (1.29x)            | 634531 KB/sec
(1.68x) |
| 8 | 410114 KB/s  | 474560 KB/sec (1.15x) |  476530 KB/sec (1.16x)            | 632547 KB/sec
(1.54x) |

(8-threaded case now 3.39x vs 0.20.2!)
                
> Simplify BlockReader to not inherit from FSInputChecker
> -------------------------------------------------------
>
>                 Key: HDFS-2129
>                 URL: https://issues.apache.org/jira/browse/HDFS-2129
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>          Components: hdfs client, performance
>            Reporter: Todd Lipcon
>            Assignee: Todd Lipcon
>             Fix For: 0.24.0
>
>         Attachments: hdfs-2129-benchmark.png, hdfs-2129.txt, hdfs-2129.txt, hdfs-2129.txt,
hdfs-2129.txt, hdfs-2129.txt, hdfs-2129.txt, hdfs-2129.txt, seq-read-1gb-bench.png
>
>
> BlockReader is currently quite complicated since it has to conform to the FSInputChecker
inheritance structure. It would be much simpler to implement it standalone. Benchmarking indicates
it's slightly faster, as well.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message