Mailing-List: contact hdfs-issues-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: hdfs-issues@hadoop.apache.org
Message-ID: <609982208.39821264520554748.JavaMail.jira@brutus.apache.org>
Date: Tue, 26 Jan 2010 15:42:34 +0000 (UTC)
From: "Wang Xu (JIRA)" <jira@apache.org>
To: hdfs-issues@hadoop.apache.org
Subject: [jira] Commented: (HDFS-708) A stress-test tool for HDFS.
In-Reply-To: <1458130731.1255576351275.JavaMail.jira@brutus>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit


    [ https://issues.apache.org/jira/browse/HDFS-708?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12805065#action_12805065 ] 

Wang Xu commented on HDFS-708:
------------------------------

I post our design illustration here.
http://gnawux.info/hadoop/2010/01/a-simple-hdfs-performance-test-tool/

And I will post the code on google code or other place tomorrow.

In our test program, synchronizer is a server written in python, it accepts the request of test program running in test nodes. Having received requests from all nodes, it admits them start pressure simultaneously.

The test program is written in Java, and it starts several threads to write or read with DFSClient. All the pressure thread record the data it has written in a variable and the main thread of the test program collect them periodically, then written into a XML file.

Analyzing the xml output file, we can tell the performance of reading and writing.

In our test program, it supports read only, write only and read-write. And it can be set as read files writen by itself or random files.

> A stress-test tool for HDFS.
> ----------------------------
>
>                 Key: HDFS-708
>                 URL: https://issues.apache.org/jira/browse/HDFS-708
>             Project: Hadoop HDFS
>          Issue Type: New Feature
>          Components: test, tools
>    Affects Versions: 0.22.0
>            Reporter: Konstantin Shvachko
>             Fix For: 0.22.0
>
>
> It would be good to have a tool for automatic stress testing HDFS, which would provide IO-intensive load on HDFS cluster.
> The idea is to start the tool, let it run overnight, and then be able to analyze possible failures.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.