hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Michael Noll (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (MAPREDUCE-2398) MRBench: setting the baseDir parameter has no effect
Date Mon, 21 Mar 2011 15:50:08 GMT

    [ https://issues.apache.org/jira/browse/MAPREDUCE-2398?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13009184#comment-13009184

Michael Noll commented on MAPREDUCE-2398:

Hmm. From what I see in the [Hudson test results|https://hudson.apache.org/hudson/job/PreCommit-MAPREDUCE-Build/142//testReport/],
the two failed tests ([org.apache.hadoop.mapred.TestFairSchedulerSystem.testFairSchedulerSystem|https://hudson.apache.org/hudson/job/PreCommit-MAPREDUCE-Build/142//testReport/org.apache.hadoop.mapred/TestFairSchedulerSystem/testFairSchedulerSystem/]
and [org.apache.hadoop.raid.TestBlockFixer.testMaxPendingFiles|https://hudson.apache.org/hudson/job/PreCommit-MAPREDUCE-Build/142//testReport/org.apache.hadoop.raid/TestBlockFixer/testMaxPendingFiles/])
have nothing to do with this patch. Or am I missing something?

On a related note, what is the current approach/policy to write proper unit tests for benchmark
classes like MRBench? In the example of this ticket, how should the unit test perform checking
whether MRBench picks up the {{-baseDir}} parameter properly from the command line? By checking
variable assignments (= events inside the code), or by checking the directory actually created
and subsequently used on HDFS (= events outside the code)?

> MRBench: setting the baseDir parameter has no effect
> ----------------------------------------------------
>                 Key: MAPREDUCE-2398
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-2398
>             Project: Hadoop Map/Reduce
>          Issue Type: Bug
>          Components: benchmarks
>    Affects Versions: 0.20.2, 0.22.0
>            Reporter: Michael Noll
>            Priority: Minor
>             Fix For: 0.22.0
>         Attachments: MAPREDUCE-2398-trunk.patch, MAPREDUCE-2398_0.20.2.patch
> The optional {{-baseDir}} parameter lets user specify the base DFS path for output/input
of MRBench.
> However, the two private variables {{INPUT_DIR}} and {{OUTPUT_DIR}} (MRBench.java) are
not updated in the case that the default value of  {{-baseDir}} is actually overwritten by
the user. Hence any input and output is always written to the default locations ({{/benchmarks/MRBench/...}}),
even though the user-supplied location for {{-baseDir}} is created (and eventually deleted
again) on HDFS.
> The bug affects at least Hadoop 0.20.2 and the current trunk (r1082703) as of March 21,

This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

View raw message