hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Konstantin Shvachko (JIRA)" <j...@apache.org>
Subject [jira] Updated: (MAPREDUCE-1832) Support for file sizes less than 1MB in DFSIO benchmark.
Date Wed, 02 Jun 2010 02:15:39 GMT

     [ https://issues.apache.org/jira/browse/MAPREDUCE-1832?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Konstantin Shvachko updated MAPREDUCE-1832:
-------------------------------------------

    Attachment: TestDFSIO-fsize.patch

- This patch lets specify files in bytes, KB, MB, GB, and TB. That is, one can say {{-fileSize
37B}}. 
The default units are still MBs. That is, {{-fileSize 37}} sets file size to 37MB.
- {{TestDFSIO}} implements {{Tool}} class, which lets it accept generic options, including
-D.
This particularly solves the problem reported in MAPREDUCE-1614. To specify output directory
do this:
{code}
TestDFSIO -Dtest.build.data=/user/me/DFSIO -write -nrFiles 12 -fileSize 1024KB
{code}
- Usage is updated respectively.
- Implementing {{Tool}} places all the main startup logic in {{run()}}, which eliminates the
necessity for methods to remain static. I removed static.

> Support for file sizes less than 1MB in DFSIO benchmark.
> --------------------------------------------------------
>
>                 Key: MAPREDUCE-1832
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-1832
>             Project: Hadoop Map/Reduce
>          Issue Type: Improvement
>          Components: benchmarks
>    Affects Versions: 0.20.2
>            Reporter: Konstantin Shvachko
>            Assignee: Konstantin Shvachko
>             Fix For: 0.20.3
>
>         Attachments: TestDFSIO-fsize.patch
>
>
> Currently DFSIO benchmark allows to specify files sizes in 1MB increments. It would be
useful to be able to specify smaller sizes.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message