hadoop-hdfs-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Amith D K <amit...@huawei.com>
Subject RE: test HDFS
Date Tue, 24 Apr 2012 06:49:21 GMT
U can check

TerraGen ,TerraSort,TerraValidate
MRBench(For MapReduce)

Above are very well known tools.

bin/hadoop jar hadoop-*test*.jar
# change to your Hadoop installation directory;
# if you followed my Hadoop tutorials, this is /usr/local/hadoop
$ cd /usr/local/hadoop
$ bin/hadoop jar hadoop-*test*.jar
An example program must be given as the first argument.
Valid program names are:
  DFSCIOTest: Distributed i/o benchmark of libhdfs.
  DistributedFSCheck: Distributed checkup of the file system consistency.
  MRReliabilityTest: A program that tests the reliability of the MR framework by injecting
  TestDFSIO: Distributed i/o benchmark.
  dfsthroughput: measure hdfs throughput
  filebench: Benchmark SequenceFile(Input|Output)Format (block,record compressed and uncompressed),
Text(Input|Output)Format (compressed and uncompressed)
  loadgen: Generic map/reduce load generator
  mapredtest: A map/reduce test check.
  mrbench: A map/reduce benchmark that can create many small jobs
  nnbench: A benchmark that stresses the namenode.
  testarrayfile: A test for flat files of binary key/value pairs.
  testbigmapoutput: A map/reduce program that works on a very big non-splittable file and
does identity map/reduce
  testfilesystem: A test for FileSystem read/write.
  testipc: A test for ipc.
  testmapredsort: A map/reduce program that validates the map-reduce framework's sort.
  testrpc: A test for rpc.
  testsequencefile: A test for flat files of binary key value pairs.
  testsequencefileinputformat: A test for sequence file input format.
  testsetfile: A test for flat files of binary key/value pairs.
  testtextinputformat: A test for text input format.
  threadedmapbench: A map/reduce benchmark that compares the performance of maps with multiple
spills over maps with 1 spill

Thanks and Regards
From: Joe Bounour [jbounour@ddn.com]
Sent: Tuesday, April 24, 2012 2:30 PM
To: hdfs-dev@hadoop.apache.org; chengyao
Subject: RE: test HDFS

Have you looked at TestDFSIO benchmark

-----Original Message-----
From: chengyao [mailto:chengyao@inspur.com]
Sent: Monday, April 23, 2012 11:22 PM
To: hdfs-dev
Subject: test HDFS

I want to test hdfs. what tools should i choose? can you tell me ? thank you.


       mail: chengyao@inspur.com
View raw message