hadoop-mapreduce-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From <stanley....@emc.com>
Subject RE: FW: NNbench and MRBench
Date Sun, 08 May 2011 02:33:23 GMT
Thanks, Marcos,

Through these links, I still can't find anything about the NNbench and MRBench.

-----Original Message-----
From: Marcos Ortiz [mailto:mlortiz@uci.cu] 
Sent: 2011年5月8日 10:23
To: mapreduce-user@hadoop.apache.org
Cc: Shi, Stanley
Subject: Re: FW: NNbench and MRBench

El 5/7/2011 8:53 PM, stanley.shi@emc.com escribió:
> Hi guys,
>
> I have a cluster of 16 machines running Hadoop. Now I want to do some benchmark on this
cluster with the "nnbench" and "mrbench".
> I'm new to the hadoop thing and have no one to refer to. I don't know what the supposed
result should I have?
> Now for mrbench, I have an average time of 22sec for a one map job. Is this too bad?
What the supposed results might be?
>
> For nnbench, what's the supposed results? Below is my result.
> ================
>                             Date&  time: 2011-05-05 20:40:25,459
>
>                          Test Operation: rename
>                              Start time: 2011-05-05 20:40:03,820
>                             Maps to run: 1
>                          Reduces to run: 1
>                      Block Size (bytes): 1
>                          Bytes to write: 0
>                      Bytes per checksum: 1
>                         Number of files: 10000
>                      Replication factor: 1
>              Successful file operations: 10000
>
>          # maps that missed the barrier: 0
>                            # exceptions: 0
>
>                             TPS: Rename: 1763
>              Avg Exec time (ms): Rename: 0.5672
>                    Avg Lat (ms): Rename: 0.4844
> null
>
>                   RAW DATA: AL Total #1: 4844
>                   RAW DATA: AL Total #2: 0
>                RAW DATA: TPS Total (ms): 5672
>         RAW DATA: Longest Map Time (ms): 5672.0
>                     RAW DATA: Late maps: 0
>               RAW DATA: # of exceptions: 0
> =============================
> One more question, when I set maps number to bigger, I get all zeros results:
> =============================
> Test Operation: create_write
>                              Start time: 2011-05-03 23:22:39,239
>                             Maps to run: 160
>                          Reduces to run: 160
>                      Block Size (bytes): 1
>                          Bytes to write: 0
>                      Bytes per checksum: 1
>                         Number of files: 1
>                      Replication factor: 1
>              Successful file operations: 0
>
>          # maps that missed the barrier: 0
>                            # exceptions: 0
>
>                 TPS: Create/Write/Close: 0
> Avg exec time (ms): Create/Write/Close: 0.0
>              Avg Lat (ms): Create/Write: NaN
>                     Avg Lat (ms): Close: NaN
>
>                   RAW DATA: AL Total #1: 0
>                   RAW DATA: AL Total #2: 0
>                RAW DATA: TPS Total (ms): 0
>         RAW DATA: Longest Map Time (ms): 0.0
>                     RAW DATA: Late maps: 0
>               RAW DATA: # of exceptions: 0
> =====================
>
> Can anyone point me to some documents?
> I really appreciate your help :)
>
> Thanks,
> stanley
>    
You can use these resources:
http://www.michael-noll.com/blog/2011/04/09/benchmarking-and-stress-testing-an-hadoop-cluster-with-terasort-testdfsio-nnbench-mrbench/
http://answers.oreilly.com/topic/460-how-to-benchmark-a-hadoop-cluster/
http://wiki.apache.org/hadoop/HardwareBenchmarks
http://www.quora.com/Apache-Hadoop/Are-there-any-good-Hadoop-benchmark-problems

Regards

-- 
Marcos Luís Ortíz Valmaseda
  Software Engineer (Large-Scaled Distributed Systems)
  University of Information Sciences,
  La Habana, Cuba
  Linux User # 418229
  http://about.me/marcosortiz


Mime
View raw message