hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Steve Loughran (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HADOOP-6332) Large-scale Automated Test Framework
Date Thu, 05 Nov 2009 11:23:32 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-6332?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12773896#action_12773896
] 

Steve Loughran commented on HADOOP-6332:
----------------------------------------

@Arun -pushing out configurations to clusters partially explores the config space, but not
very broadly; more leading edge tricks involve machine generation of very different configurations,
and/or pseudo-RNG driven configuration option generation

Some videos on this topic

#Skoll: Distributed Continuous QA
http://www.cs.umd.edu/~atif/papers/MemonICSE2004.pdf 
http://video.google.ca/videoplay?docid=8839342624264709864

# How we test -these are tests that run under junit from Ant/IDE, but can then bring up a
cluster and run junit underneath. It gets complex
http://www.youtube.com/watch?v=NKshZGUWHJ4

So, while I agree, you do need ways to bring up clusters -indeed, I have some I can demo,
I do think it can be best done outside the junit test run itself
# Ant tasks to allocate machines from different IaaS systems -that includes selecting from
a list of physical machines you have to hand. 
# whatever we use to explore the configuration space runs very differently from inside a Junit
test run, because you want to create clusters with different options, *then run the entire
test suite*. What is key is to get the output from that run and merge it with everything else.

Like I said, we should have a phone conf about this before anyone starts coding, I'd like
to see what Alex has done and I can show what I have, I'd like to hear from Stephen about
how IBM run their tests too. How about everyone who is at apachecon meet up and talk about
this, and then next week we can have an online gettogether in some timezone that works for
everyone?




> Large-scale Automated Test Framework
> ------------------------------------
>
>                 Key: HADOOP-6332
>                 URL: https://issues.apache.org/jira/browse/HADOOP-6332
>             Project: Hadoop Common
>          Issue Type: New Feature
>          Components: test
>            Reporter: Arun C Murthy
>            Assignee: Arun C Murthy
>             Fix For: 0.21.0
>
>
> Hadoop would benefit from having a large-scale, automated, test-framework. This jira
is meant to be a master-jira to track relevant work.
> ----
> The proposal is a junit-based, large-scale test framework which would run against _real_
clusters.
> There are several pieces we need to achieve this goal:
> # A set of utilities we can use in junit-based tests to work with real, large-scale hadoop
clusters. E.g. utilities to bring up to deploy, start & stop clusters, bring down tasktrackers,
datanodes, entire racks of both etc.
> # Enhanced control-ability and inspect-ability of the various components in the system
e.g. daemons such as namenode, jobtracker should expose their data-structures for query/manipulation
etc. Tests would be much more relevant if we could for e.g. query for specific states of the
jobtracker, scheduler etc. Clearly these apis should _not_ be part of the production clusters
- hence the proposal is to use aspectj to weave these new apis to debug-deployments.
> ----
> Related note: we should break up our tests into at least 3 categories:
> # src/test/unit -> Real unit tests using mock objects (e.g. HDFS-669 & MAPREDUCE-1050).
> # src/test/integration -> Current junit tests with Mini* clusters etc.
> # src/test/system -> HADOOP-6332 and it's children

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message