hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Konstantin Shvachko (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (MAPREDUCE-4651) Benchmarking random reads with DFSIO
Date Wed, 12 Sep 2012 08:42:08 GMT

     [ https://issues.apache.org/jira/browse/MAPREDUCE-4651?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Konstantin Shvachko updated MAPREDUCE-4651:
-------------------------------------------

    Attachment: randomDFSIO.patch

The patch
# Introduces the three types of random reads.
# It also adds getIOStream() method, which excludes stream construction from the timed part
of the execution. This is important for small writes and reads.
And this let me move all compression functionality from IOMapperBase to TestDFSIO, where it
truly belongs.
# Converted to JUnit 4 format. Finally. And added test cases for new benchmarks.
# Fixed couple of warnings and removed unnecessary generic parameters.
                
> Benchmarking random reads with DFSIO
> ------------------------------------
>
>                 Key: MAPREDUCE-4651
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-4651
>             Project: Hadoop Map/Reduce
>          Issue Type: New Feature
>          Components: benchmarks, test
>    Affects Versions: 1.0.0
>            Reporter: Konstantin Shvachko
>            Assignee: Konstantin Shvachko
>         Attachments: randomDFSIO.patch
>
>
> TestDFSIO measures throughput of HDFS write, read, and append operations. It will be
useful to have an option to use it for benchmarking random reads.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message