hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ravi Gummadi (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HADOOP-4191) Add a testcase for jobhistory
Date Tue, 10 Feb 2009 18:45:03 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-4191?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12672356#action_12672356

Ravi Gummadi commented on HADOOP-4191:

Hi Amar,
  FAILED_MAPS, FAILED_REDUCES are logged only from logFinished() --- i.e. for a successful
job for which all other job level keys are validated. 
Is the following code being added to validateJobLevelKJeys() look fine ? (looks like too much
dependant on the fact that validateJobLevelKeys() is called only after UtilsForTests.runJobSucceed()
and that internally calls UtilsForTests.runJob(). Hmm may be we can send expected number of
finished_maps, finished_reduces, failed_maps, failed_reduces as parameters ???)

    if (status.equals("SUCCESS")) {
      // Since UtilsForTests.runJob() sets number of maps to be run as 1 and
      // number of reduces to be run as 0, we expect the following values
      // for the case of runJobSucceed()
      String fin_maps = values.get(Keys.FINISHED_MAPS);
      assertTrue("Unexpected number of finished maps in history file",
                 fin_maps == 1);
      String fin_reduces = values.get(Keys.FINISHED_REDUCES);
      assertTrue("Unexpected number of finished reduces in history file",
                 fin_reduces == 0);
      String failed_maps = values.get(Keys.FAILED_MAPS);
      assertTrue("Unexpected number of failed maps in history file",
                 failed_maps == 0);
      String failed_reduces = values.get(Keys.FAILED_REDUCES);
      assertTrue("Unexpected number of finished reduces in history file",
                 failed_reduces == 0);


> Add a testcase for jobhistory
> -----------------------------
>                 Key: HADOOP-4191
>                 URL: https://issues.apache.org/jira/browse/HADOOP-4191
>             Project: Hadoop Core
>          Issue Type: Improvement
>          Components: mapred, test
>            Reporter: Amar Kamat
>            Assignee: Ravi Gummadi
>         Attachments: HADOOP-4191.patch, HADOOP-4191.v2.patch
> Changes in job history might break the history parser which in turn might break some
features like jobtracker-recovery, history-viewer etc. There should be a testcase that catches
these incompatible changes early and informs about the expected change.

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message