hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Mark Kerzner <mark.kerz...@shmsoft.com>
Subject Re: How to test the performance of NN?
Date Fri, 07 Jun 2013 01:41:19 GMT
Awesome advice, Ivan! It works for me now.

Thank you,
Mark


On Thu, Jun 6, 2013 at 12:18 AM, Ivan Mitic <ivanmi@microsoft.com> wrote:

>  Hi Mark,****
>
> ** **
>
> Your NNBench output does not look OK to me. Below is an example NNBench
> cmd line I used some time ago and its output.****
>
> ** **
>
> Btw, what I found useful  in testing the disk IO performance for namenode
> operations (others might find this unusual) is the TestEditLog unittest. I
> would modify the test as appropriate and run the simulation. This was a
> quick micro-benchmark I used to compare a couple of devices. Note that
> you’ll have to modify the test in trunk and remove the line with
> EditLogFileOutputStream.setShouldSkipFsyncForTesting(), otherwise you’re
> not testing the real thing.****
>
> ** **
>
> ** **
>
> hadoop.cmd jar hadoop-test-1.1.0-SNAPSHOT.jar nnbench -operation
> create_write -maps 32 -reduces 1 -blockSize 1 -bytesToWrite 20
> -bytesPerChecksum 1 -numberOfFiles 500 -replicationFactorPerFile 1****
>
> ** **
>
> ** **
>
> -------------- NNBench -------------- :****
>
>                                 Version: NameNode Benchmark 0.4****
>
>                             Date & time: 2012-12-18 07:29:20,832****
>
> ** **
>
>                          Test Operation: create_write****
>
>                              Start time: 2012-12-18 07:25:36,111****
>
>                             Maps to run: 32****
>
>                          Reduces to run: 1****
>
>                      Block Size (bytes): 1****
>
>                          Bytes to write: 20****
>
>                      Bytes per checksum: 1****
>
>                         Number of files: 500****
>
>                      Replication factor: 1****
>
>              Successful file operations: 16000****
>
> ** **
>
>          # maps that missed the barrier: 0****
>
>                            # exceptions: 0****
>
> ** **
>
>                 TPS: Create/Write/Close: 153****
>
> Avg exec time (ms): Create/Write/Close: 301.6148125****
>
>              Avg Lat (ms): Create/Write: 68.25625****
>
>                     Avg Lat (ms): Close: 233.245****
>
> ** **
>
>                   RAW DATA: AL Total #1: 1092100****
>
>                   RAW DATA: AL Total #2: 3731920****
>
>                RAW DATA: TPS Total (ms): 4825837****
>
>         RAW DATA: Longest Map Time (ms): 208692.0****
>
>                     RAW DATA: Late maps: 0****
>
>               RAW DATA: # of exceptions: 0****
>
> ** **
>
> Hope this helps,****
>
> Ivan****
>
> ** **
>
> *From:* Mark Kerzner [mailto:mark.kerzner@shmsoft.com]
> *Sent:* Wednesday, June 05, 2013 5:16 PM
> *To:* Hadoop User
> *Subject:* Re: How to test the performance of NN?****
>
> ** **
>
> Hi, Ivan,****
>
> thank you for your willingness to help. First, what do I mean by a more
> efficient NN? - I am experimenting with Kove <http://kove.com/> device,
> persistent memory. I want to do one of the two things:****
>
> 1. Use it for FSDirectory, with the expectation of very fast writes, to
> see if this improves performance.****
>
> 2. Use it for BlockMaps, not for performance, but for fault-tolerance.****
>
> To prepare for this, I want to run some benchmarks on the given install
> just to get experience running it. Then I will run it on a 3-node cluster
> without Kove, and then with Kove.****
>
> Here is what I got from nnbench, and I am trying to find what I should
> watch for to measure NN performance. Does this all make sense?****
>
> Thank you,
> Mark
>
>
> 13/06/05 16:50:16 INFO hdfs.NNBench: -------------- NNBench --------------
> :
> 13/06/05 16:50:16 INFO hdfs.NNBench:
> Version: NameNode Benchmark 0.4
> 13/06/05 16:50:16 INFO hdfs.NNBench:                            Date &
> time: 2013-06-05 16:50:16,412
> 13/06/05 16:50:16 INFO hdfs.NNBench:
> 13/06/05 16:50:16 INFO hdfs.NNBench:                         Test
> Operation: open_read
> 13/06/05 16:50:16 INFO hdfs.NNBench:                             Start
> time: 2013-06-05 16:48:34,792
> 13/06/05 16:50:16 INFO hdfs.NNBench:                            Maps to
> run: 12
> 13/06/05 16:50:16 INFO hdfs.NNBench:                         Reduces to
> run: 6
> 13/06/05 16:50:16 INFO hdfs.NNBench:                     Block Size
> (bytes): 1
> 13/06/05 16:50:16 INFO hdfs.NNBench:                         Bytes to
> write: 0
> 13/06/05 16:50:16 INFO hdfs.NNBench:                     Bytes per
> checksum: 1
> 13/06/05 16:50:16 INFO hdfs.NNBench:                        Number of
> files: 1000
> 13/06/05 16:50:16 INFO hdfs.NNBench:                     Replication
> factor: 3
> 13/06/05 16:50:16 INFO hdfs.NNBench:             Successful file
> operations: 0
> 13/06/05 16:50:16 INFO hdfs.NNBench:
> 13/06/05 16:50:16 INFO hdfs.NNBench:         # maps that missed the
> barrier: 0
> 13/06/05 16:50:16 INFO hdfs.NNBench:                           #
> exceptions: 0
> 13/06/05 16:50:16 INFO hdfs.NNBench:
> 13/06/05 16:50:16 INFO hdfs.NNBench:                         TPS:
> Open/Read: 0
> 13/06/05 16:50:16 INFO hdfs.NNBench:          Avg Exec time (ms):
> Open/Read: 0.0
> 13/06/05 16:50:16 INFO hdfs.NNBench:                     Avg Lat (ms):
> Open: Infinity
> 13/06/05 16:50:16 INFO hdfs.NNBench:                   Avg Lat (ms): Read:
> NaN
> 13/06/05 16:50:16 INFO hdfs.NNBench:
> 13/06/05 16:50:16 INFO hdfs.NNBench:                  RAW DATA: AL Total
> #1: 4665
> 13/06/05 16:50:16 INFO hdfs.NNBench:                  RAW DATA: AL Total
> #2: 0
> 13/06/05 16:50:16 INFO hdfs.NNBench:               RAW DATA: TPS Total
> (ms): 0
> 13/06/05 16:50:16 INFO hdfs.NNBench:        RAW DATA: Longest Map Time
> (ms): 0.0
> 13/06/05 16:50:16 INFO hdfs.NNBench:                    RAW DATA: Late
> maps: 0
> 13/06/05 16:50:16 INFO hdfs.NNBench:              RAW DATA: # of
> exceptions: 0****
>
> ** **
>
> ** **
>
> On Wed, Jun 5, 2013 at 4:32 PM, Ivan Mitic <ivanmi@microsoft.com> wrote:**
> **
>
> Hi Mark,****
>
>  ****
>
> NNBench is a namenode load test. Output of the test is the set of
> performance numbers, like transactions per second, average latency of
> operations, etc.****
>
>  ****
>
> What do you mean by trying to create a more efficient namenode? What
> dimension are you trying to optimize? Depending on this, people on this
> list might be able to guide you to something more appropriate. ****
>
>  ****
>
> Hope this helps,****
>
> Ivan  ****
>
>  ****
>
> *From:* Mark Kerzner [mailto:mark.kerzner@shmsoft.com]
> *Sent:* Wednesday, June 05, 2013 3:17 PM
> *To:* Hadoop User
> *Subject:* How to test the performance of NN?****
>
>  ****
>
> Hi,
>
> I am trying to create a more efficient namenode, and for that I need to
> the standard distribution, and then compare it to my version.
>
> Which benchmark should I run? I am doing nnbench, but it is not telling me
> anything about performance, only about potential failures.
>
> Thank you.****
>
> Sincerely,
> Mark****
>
> ** **
>

Mime
View raw message