hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Konstantin Boudnik (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HADOOP-6332) Large-scale Automated Test Framework
Date Wed, 09 Dec 2009 19:44:18 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-6332?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12788261#action_12788261

Konstantin Boudnik commented on HADOOP-6332:

Thanks for the answers, Sharad. All of it makes sense. One comment though:

bq. But seems like tests will benefit by having a control on start/stop daemons (for example
to test lost/blacklisted TT, tests may want to kill a TT). How and which tar balls are pushed
and deployed are not in scope of this because test cases need not bother about it.

Right, actual bits push has to be done somewhere else: Hudson or else.

bq. To work with already started cluster, a config flag something like NO_CLUSTER_START can
be set which will let test suites skip the cluster start/stop step.
My thought on this was that the cluster's component restart part should be done in a way consistent
with setup/teardown approach of pretty much any test framework like JUnit. If a test needs
to start/stop a cluster then it needs to specify {{@Before}} and {{@After}} methods which
will do that using provided control primitives (e.g. start_datanode.sh, stop_datanode.sh or

> Large-scale Automated Test Framework
> ------------------------------------
>                 Key: HADOOP-6332
>                 URL: https://issues.apache.org/jira/browse/HADOOP-6332
>             Project: Hadoop Common
>          Issue Type: New Feature
>          Components: test
>            Reporter: Arun C Murthy
>            Assignee: Arun C Murthy
>             Fix For: 0.21.0
>         Attachments: 6332_v1.patch
> Hadoop would benefit from having a large-scale, automated, test-framework. This jira
is meant to be a master-jira to track relevant work.
> ----
> The proposal is a junit-based, large-scale test framework which would run against _real_
> There are several pieces we need to achieve this goal:
> # A set of utilities we can use in junit-based tests to work with real, large-scale hadoop
clusters. E.g. utilities to bring up to deploy, start & stop clusters, bring down tasktrackers,
datanodes, entire racks of both etc.
> # Enhanced control-ability and inspect-ability of the various components in the system
e.g. daemons such as namenode, jobtracker should expose their data-structures for query/manipulation
etc. Tests would be much more relevant if we could for e.g. query for specific states of the
jobtracker, scheduler etc. Clearly these apis should _not_ be part of the production clusters
- hence the proposal is to use aspectj to weave these new apis to debug-deployments.
> ----
> Related note: we should break up our tests into at least 3 categories:
> # src/test/unit -> Real unit tests using mock objects (e.g. HDFS-669 & MAPREDUCE-1050).
> # src/test/integration -> Current junit tests with Mini* clusters etc.
> # src/test/system -> HADOOP-6332 and it's children

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message