hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Konstantin Shvachko (JIRA)" <j...@apache.org>
Subject [jira] Updated: (MAPREDUCE-1832) Support for file sizes less than 1MB in DFSIO benchmark.
Date Wed, 02 Jun 2010 02:15:39 GMT

     [ https://issues.apache.org/jira/browse/MAPREDUCE-1832?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel

Konstantin Shvachko updated MAPREDUCE-1832:

    Attachment: TestDFSIO-fsize.patch

- This patch lets specify files in bytes, KB, MB, GB, and TB. That is, one can say {{-fileSize
The default units are still MBs. That is, {{-fileSize 37}} sets file size to 37MB.
- {{TestDFSIO}} implements {{Tool}} class, which lets it accept generic options, including
This particularly solves the problem reported in MAPREDUCE-1614. To specify output directory
do this:
TestDFSIO -Dtest.build.data=/user/me/DFSIO -write -nrFiles 12 -fileSize 1024KB
- Usage is updated respectively.
- Implementing {{Tool}} places all the main startup logic in {{run()}}, which eliminates the
necessity for methods to remain static. I removed static.

> Support for file sizes less than 1MB in DFSIO benchmark.
> --------------------------------------------------------
>                 Key: MAPREDUCE-1832
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-1832
>             Project: Hadoop Map/Reduce
>          Issue Type: Improvement
>          Components: benchmarks
>    Affects Versions: 0.20.2
>            Reporter: Konstantin Shvachko
>            Assignee: Konstantin Shvachko
>             Fix For: 0.20.3
>         Attachments: TestDFSIO-fsize.patch
> Currently DFSIO benchmark allows to specify files sizes in 1MB increments. It would be
useful to be able to specify smaller sizes.

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message