Mailing-List: contact hdfs-issues-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: hdfs-issues@hadoop.apache.org
Message-ID: <905488.26511272349235493.JavaMail.jira@thor>
Date: Tue, 27 Apr 2010 02:20:35 -0400 (EDT)
From: "Joshua Harlow (JIRA)" <jira@apache.org>
To: hdfs-issues@hadoop.apache.org
Subject: [jira] Updated: (HDFS-708) A stress-test tool for HDFS.
In-Reply-To: <1458130731.1255576351275.JavaMail.jira@brutus>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit


     [ https://issues.apache.org/jira/browse/HDFS-708?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Joshua Harlow updated HDFS-708:
-------------------------------

          Status: Patch Available  (was: Open)
    Release Note: 
Does not currently provide anything but uniform distribution. 
Uses some older depreciated class interfaces (for mapper and reducer)
This was tested on 0.20 and 0.22 (locally) so it should be fairly backwards compatible.

This utility provides the basic features for the defined feature.
It allows a set of mappers to run which will perform various operations on a file system with various distributions and ratios.
It implements most of the features provided in the given specification (minus the different distributions) and has been tested on a individual machine (using a local file system) and a single & multinode cluster (using a hdfs filesystem).

It can be ran by building the test jar file and running -help.
Through a command such as:
hadoop org.apache.hadoop.fs.slive.SliveTest -help

A sample run can be performed by doing the following:
hadoop org.apache.hadoop.fs.slive.SliveTest -create 100 -files 1000 -ops 500 -duration 1  -maps 1   -baseDir /XYZ


> A stress-test tool for HDFS.
> ----------------------------
>
>                 Key: HDFS-708
>                 URL: https://issues.apache.org/jira/browse/HDFS-708
>             Project: Hadoop HDFS
>          Issue Type: New Feature
>          Components: test, tools
>    Affects Versions: 0.22.0
>            Reporter: Konstantin Shvachko
>             Fix For: 0.22.0
>
>         Attachments: SLiveTest.pdf
>
>
> It would be good to have a tool for automatic stress testing HDFS, which would provide IO-intensive load on HDFS cluster.
> The idea is to start the tool, let it run overnight, and then be able to analyze possible failures.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.