hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Todd Lipcon (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (HDFS-2129) Simplify BlockReader to not inherit from FSInputChecker
Date Fri, 05 Aug 2011 17:59:29 GMT

     [ https://issues.apache.org/jira/browse/HDFS-2129?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Todd Lipcon updated HDFS-2129:
------------------------------

    Attachment: hdfs-2129-benchmark.png
                hdfs-2129.txt

Attaching updated patch and benchmark.
The benchmark is from running TestParallelRead ten times each with the patch and without the
patch, and throwing away the first two runs to ignore JVM warmup. The patch provides a modest
improvement for 4 and 8 concurrent reader threads, and no significant difference for 16. This
makes sense since the test machine has only 4 physical cores plus hyperthreading.

> Simplify BlockReader to not inherit from FSInputChecker
> -------------------------------------------------------
>
>                 Key: HDFS-2129
>                 URL: https://issues.apache.org/jira/browse/HDFS-2129
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>          Components: hdfs client
>            Reporter: Todd Lipcon
>            Assignee: Todd Lipcon
>             Fix For: 0.23.0
>
>         Attachments: hdfs-2129-benchmark.png, hdfs-2129.txt, hdfs-2129.txt
>
>
> BlockReader is currently quite complicated since it has to conform to the FSInputChecker
inheritance structure. It would be much simpler to implement it standalone. Benchmarking indicates
it's slightly faster, as well.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message