hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Nigel Daley (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HADOOP-763) NameNode benchmark using mapred is insufficient
Date Thu, 30 Nov 2006 22:33:22 GMT
    [ http://issues.apache.org/jira/browse/HADOOP-763?page=comments#action_12454769 ] 
Nigel Daley commented on HADOOP-763:

Elaborating on why running the namenode benchmark as a map/reduce is problematic, if a map
fails, you either don't want it to rerun (not currently possible) or you want to detect a
rerun map and disregard its output (not easily possible).  

> NameNode benchmark using mapred is insufficient
> -----------------------------------------------
>                 Key: HADOOP-763
>                 URL: http://issues.apache.org/jira/browse/HADOOP-763
>             Project: Hadoop
>          Issue Type: Improvement
>          Components: test
>    Affects Versions: 0.8.0
>            Reporter: Nigel Daley
>         Assigned To: Nigel Daley
>            Priority: Minor
>         Attachments: HADOOP-763.patch
> The current namenode benchmark (org.apache.hadoop.examples.NNBench) uses map/reduce to
distribute a load on the namenode.  For the purposes of loading the namenode, this model gives
insufficient control over job start and failure recovery.  I propose the namenode benchmark
be re-written to use slaves.sh directly to execute the namenode benchmark.  The benchmark
should also give finer control over the operations executed and the timings reported.

This message is automatically generated by JIRA.
If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira


View raw message