hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Vinod K V (JIRA)" <j...@apache.org>
Subject [jira] Updated: (HADOOP-4830) Have end to end tests based on MiniMRCluster to verify that queue capacities are honoured.
Date Mon, 22 Dec 2008 13:02:44 GMT

     [ https://issues.apache.org/jira/browse/HADOOP-4830?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel

Vinod K V updated HADOOP-4830:

    Attachment: HADOOP-4830-20081222-svn.2

Attaching patch. This includes
 - o.a.h.mapred.ControlledMapReduceJob to run a job whose tasks' execution can be precisely
controlled by a user.
 - o.a.h.mapred.ClusterWithCapacityScheduler to start a MiniMRCluster with CapacityScheduler
configured as the scheduler. It provides api for configuring both the cluster as well as the
scheduler. Any test to verify behaviour of CapacityTaskScheduler should extend this class.
 - Tests o.a.h.m.TestControlledMapReduceJob and o.a.h.m.TestClusterWithCapacityScheduler to
test both the above utility classes. Later is very minimal and can be improved as we go.
 - o.a.h.m.TestQueueCapacities to automate the tests related to queue capacities and to serve
as a demonstration of how to use the above utilities for testing CapacityTaskScheduler. This
includes basic queue capacity tests. Many other tests are possible but some of them involve
testing reclamation of slots and hence will be done in a separate Jira (HADOOP-4834).

> Have end to end tests based on MiniMRCluster to verify that queue capacities are honoured.
> ------------------------------------------------------------------------------------------
>                 Key: HADOOP-4830
>                 URL: https://issues.apache.org/jira/browse/HADOOP-4830
>             Project: Hadoop Core
>          Issue Type: Bug
>          Components: contrib/capacity-sched
>            Reporter: Vinod K V
>            Assignee: Vinod K V
>         Attachments: HADOOP-4830-20081222-svn.2
> At present, we only have unit tests that make use of FakeTaskManager and that only test
the proper functionality of capacity scheduler in isolation. Many issues unearthed recently
proved that this is not enough and that it is required to have end-to-end tests so that real
JT is brought into the picture and with that the interaction of the scheduler with JT. This
issue along with few other related jiras should automate and replace the end-to-end tests
that are now manually done by QA, using MiniMRCluster.

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message