hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Joshua Harlow (JIRA)" <j...@apache.org>
Subject [jira] Updated: (HDFS-708) A stress-test tool for HDFS.
Date Tue, 27 Apr 2010 06:20:35 GMT

     [ https://issues.apache.org/jira/browse/HDFS-708?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Joshua Harlow updated HDFS-708:
-------------------------------

          Status: Patch Available  (was: Open)
    Release Note: 
Does not currently provide anything but uniform distribution. 
Uses some older depreciated class interfaces (for mapper and reducer)
This was tested on 0.20 and 0.22 (locally) so it should be fairly backwards compatible.

This utility provides the basic features for the defined feature.
It allows a set of mappers to run which will perform various operations on a file system with
various distributions and ratios.
It implements most of the features provided in the given specification (minus the different
distributions) and has been tested on a individual machine (using a local file system) and
a single & multinode cluster (using a hdfs filesystem).

It can be ran by building the test jar file and running -help.
Through a command such as:
hadoop org.apache.hadoop.fs.slive.SliveTest -help

A sample run can be performed by doing the following:
hadoop org.apache.hadoop.fs.slive.SliveTest -create 100 -files 1000 -ops 500 -duration 1 
-maps 1   -baseDir /XYZ



> A stress-test tool for HDFS.
> ----------------------------
>
>                 Key: HDFS-708
>                 URL: https://issues.apache.org/jira/browse/HDFS-708
>             Project: Hadoop HDFS
>          Issue Type: New Feature
>          Components: test, tools
>    Affects Versions: 0.22.0
>            Reporter: Konstantin Shvachko
>             Fix For: 0.22.0
>
>         Attachments: SLiveTest.pdf
>
>
> It would be good to have a tool for automatic stress testing HDFS, which would provide
IO-intensive load on HDFS cluster.
> The idea is to start the tool, let it run overnight, and then be able to analyze possible
failures.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message