hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Alex Loddengaard (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HADOOP-6248) Circus: Proposal and Preliminary Code for a Hadoop System Testing Framework
Date Wed, 09 Sep 2009 22:21:57 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-6248?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12753299#action_12753299
] 

Alex Loddengaard commented on HADOOP-6248:
------------------------------------------

h1. Overview

This proposal defines a system testing framework for Hadoop, named Circus. Circus will let
Hadoop testers and users run one or more tests on a Hadoop "context." A context might be a
local pseudo-distributed cluster, an EC2 cluster, two Hadoop clusters-each running a different
version of Hadoop-or anything else. A test might be the canned pi example, HDFS manipulations,
an HDFS upgrade, custom MapReduce code that is compiled and checked for validity, a distcp
across two different versions of Hadoop, or anything else. This proposal will provide more
examples of contexts and jobs in its later sections.

h1. System Testing vs. Unit Testing

One might wonder how Circus is different from a unit testing framework such as JUnit.  A unit
testing framework is meant to run short-lived tests, one after another. Unit tests, by definition,
are meant to test small, independent pieces of a larger entity.  A system test, in general,
is larger than a unit test, in that a system test might run for several hours, analyze logs,
run on a large cluster of machines, or test the runtime and performance of a job. We will
see more use cases later.

h1. Motivation and Use Cases

# Hadoop users want to know if their MapReduce jobs are compatible with new Hadoop releases.
 They should have a tool that lets them easily spin up a cluster running a particular version
of Hadoop and compile and run MapReduce jobs on that cluster.  Ideally users will put their
MR jobs in Circus and put Circus in their QA pipeline.  Then, when considering an upgrade,
they can make minimal changes to test those same jobs in a newer release of Hadoop.
# Hadoop users want to know how their MapReduce jobs will perform in a new Hadoop release.
# Hadoop QA engineers want to run complicated tests such as HDFS upgrades, distcp jobs between
two different versions of HDFS, etc.  Testers want these tests to all be part of the same
framework so a suite of regression tests can be run with a single command.
# Hadoop QA engineers want to continually be running Hadoop jobs to ensure API compatibility
between versions.  Circus could power a community-driven website, where community members
upload Hadoop jobs and sample data that help generate reports on backwards compatibility,
both at the API level and at the runtime level.

h1. Expected Usage

I imagine that large users of Hadoop can port their MapReduce jobs to work in this framework,
and add Circus to their QA pipeline.  They might also configure the local context to work
with their dev cluster.  I imagine Hadoop QA engineers can use Circus to test some of the
tricky use cases of Hadoop, such as cross-version distcp jobs, HDFS upgrades, or anything
else.  Testing these use cases will require that the QA engineer write contexts and tests.

h1. Implementation Details

Please see the source code.  A README file exists that explains interfaces and other implementation
details.

h1. Open Issues and Questions

# Can this framework be useful to Hadoop developers?  And if so, how?
## Should Circus provide hooks for inducing failure in a Hadoop cluster?
## Might a trunk context be useful so developers can spin up a cluster very quickly from their
local SVN checkout?
## Can the local context allow the user to specify a core jar to be used, instead of a release?
# What other canned contexts would be useful besides the local context?  Probably an EC2 context.
# How can this framework cater to performance testing?  Is just providing time information
sufficient?
# Should tests specify the contexts they are supposed to run in?  Seems like tests should
at least have the option to.  This will be particularly interesting for tests that need to
run on two Hadoop clusters.
# Should the environment variables specified by a context be validated?  That is, should the
interface for contexts be more rigid?
# Should Circus provide a facility for analyzing the Hadoop logs generated during the run
of a test?

> Circus: Proposal and Preliminary Code for a Hadoop System Testing Framework
> ---------------------------------------------------------------------------
>
>                 Key: HADOOP-6248
>                 URL: https://issues.apache.org/jira/browse/HADOOP-6248
>             Project: Hadoop Common
>          Issue Type: New Feature
>          Components: test
>         Environment: Python, bash
>            Reporter: Alex Loddengaard
>
> This issue contains a proposal and preliminary source code for Circus, a Hadoop system
testing framework.  At a high level, Circus will help Hadoop users and QA engineers to run
system tests on a configurable Hadoop cluster, or distribution of Hadoop.  See the comment
below for the proposal itself.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message