hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Mukund Madhugiri (JIRA)" <j...@apache.org>
Subject [jira] Updated: (HADOOP-2000) Re-write NNBench to use MapReduce
Date Tue, 09 Oct 2007 18:20:50 GMT

     [ https://issues.apache.org/jira/browse/HADOOP-2000?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Mukund Madhugiri updated HADOOP-2000:
-------------------------------------

    Status: Patch Available  (was: Open)

Attaching patch for code review

Here is a sample run output:
---------------------------------------------

bin/hadoop jar hadoop-0.15.0-dev-test.jar nnbench -maps 1 -reduces 1 -startTime `date +"%s"`
-blockSize 1 -bytesPerChecksum 1 -replicationFactorPerFile 1 -numberOfFiles 100
NameNode Benchmark 0.4
07/10/09 18:14:09 INFO dfs.NNBench: Test Inputs: 
07/10/09 18:14:09 INFO dfs.NNBench:            Test Operation: all
07/10/09 18:14:09 INFO dfs.NNBench:            Number of maps: 1
07/10/09 18:14:09 INFO dfs.NNBench:         Number of reduces: 1
07/10/09 18:14:09 INFO dfs.NNBench:                Block Size: 1
07/10/09 18:14:09 INFO dfs.NNBench:        Bytes per checksum: 1
07/10/09 18:14:09 INFO dfs.NNBench:           Number of files: 100
07/10/09 18:14:09 INFO dfs.NNBench:        Replication factor: 1
07/10/09 18:14:10 INFO dfs.NNBench: Creating control files
07/10/09 18:14:10 INFO mapred.FileInputFormat: Total input paths to process : 1
07/10/09 18:14:10 INFO mapred.JobClient: Running job: job_200710032050_0199
07/10/09 18:14:11 INFO mapred.JobClient:  map 0% reduce 0%
07/10/09 18:14:17 INFO mapred.JobClient:  map 100% reduce 0%
07/10/09 18:14:18 INFO mapred.JobClient:  map 100% reduce 100%
07/10/09 18:14:19 INFO mapred.JobClient: Job complete: job_200710032050_0199
07/10/09 18:14:19 INFO mapred.JobClient: Counters: 10
07/10/09 18:14:19 INFO mapred.JobClient:   Job Counters 
07/10/09 18:14:19 INFO mapred.JobClient:     Launched map tasks=1
07/10/09 18:14:19 INFO mapred.JobClient:     Launched reduce tasks=1
07/10/09 18:14:19 INFO mapred.JobClient:     Data-local map tasks=1
07/10/09 18:14:19 INFO mapred.JobClient:   Map-Reduce Framework
07/10/09 18:14:19 INFO mapred.JobClient:     Map input records=1
07/10/09 18:14:19 INFO mapred.JobClient:     Map output records=5
07/10/09 18:14:19 INFO mapred.JobClient:     Map input bytes=39
07/10/09 18:14:19 INFO mapred.JobClient:     Map output bytes=90
07/10/09 18:14:19 INFO mapred.JobClient:     Reduce input groups=5
07/10/09 18:14:19 INFO mapred.JobClient:     Reduce input records=5
07/10/09 18:14:19 INFO mapred.JobClient:     Reduce output records=5
07/10/09 18:14:19 INFO dfs.NNBench: ------ NNBench ------ : 
07/10/09 18:14:19 INFO dfs.NNBench:                   Version: NameNode Benchmark 0.4
07/10/09 18:14:19 INFO dfs.NNBench:               Date & time: Tue Oct 09 18:14:19 UTC
2007
07/10/09 18:14:19 INFO dfs.NNBench: 
07/10/09 18:14:19 INFO dfs.NNBench:            Test Operation: all
07/10/09 18:14:19 INFO dfs.NNBench:                      Maps: 1
07/10/09 18:14:19 INFO dfs.NNBench:                   Reduces: 1
07/10/09 18:14:19 INFO dfs.NNBench:        Block Size (bytes): 1
07/10/09 18:14:19 INFO dfs.NNBench:            Bytes to write: 1
07/10/09 18:14:19 INFO dfs.NNBench:        Bytes per checksum: 1
07/10/09 18:14:19 INFO dfs.NNBench:           Number of files: 100
07/10/09 18:14:19 INFO dfs.NNBench:        Replication factor: 1
07/10/09 18:14:19 INFO dfs.NNBench: 
07/10/09 18:14:19 INFO dfs.NNBench:     TPS: Create and Close: 72
07/10/09 18:14:19 INFO dfs.NNBench:          TPS: Open (Read): 2777
07/10/09 18:14:19 INFO dfs.NNBench:               TPS: Rename: 800
07/10/09 18:14:19 INFO dfs.NNBench:               TPS: Delete: 7142
07/10/09 18:14:19 INFO dfs.NNBench: 
07/10/09 18:14:19 INFO dfs.NNBench:      Avg Lat (ms): Create: 1.35
07/10/09 18:14:19 INFO dfs.NNBench:       Avg Lat (ms): Close: 26.28
07/10/09 18:14:19 INFO dfs.NNBench: Avg Lat (ms): Open (Read): 0.36
07/10/09 18:14:19 INFO dfs.NNBench:      Avg Lat (ms): Rename: 1.25
07/10/09 18:14:19 INFO dfs.NNBench:      Avg Lat (ms): Delete: 0.14

---------------------------------------------

> Re-write NNBench to use MapReduce
> ---------------------------------
>
>                 Key: HADOOP-2000
>                 URL: https://issues.apache.org/jira/browse/HADOOP-2000
>             Project: Hadoop
>          Issue Type: Test
>          Components: test
>    Affects Versions: 0.15.0
>            Reporter: Mukund Madhugiri
>            Assignee: Mukund Madhugiri
>
> The proposal is to re-write the NNBench benchmark/test to measure Namenode operations
using MapReduce. Two buckets of measurements will be done:
> 1. Transactions per second 
> 2. Average latency
> for these operations
> - Create and Close file
> - Open file
> - Rename file
> - Delete file

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message