hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Steve Loughran (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HADOOP-6248) Circus: Proposal and Preliminary Code for a Hadoop System Testing Framework
Date Fri, 25 Sep 2009 16:00:16 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-6248?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12759597#action_12759597
] 

Steve Loughran commented on HADOOP-6248:
----------------------------------------

# I do functional testing with my smartfrog wrapper around the lifecycle-enabled version of
Hadoop. what I have not done yet -but which fully automated test frameworks can do- is have
the test framework try out different configuration options, to *explore the configuration
space*. That's fairly bleeding edge in system testing, especially when that configuration
space includes the (virtualized) network infrastructure too, but it is a fantastic way to
find bugs.

# I do think it's critical that Hadoop tests the means by which it gets started up. For {{hadoop.sh}}
that means the shell scripts need to run. For me, that means bring up some machines with the
right RPMs installed, push out the config to them, and then run functional tests against a
live cluster.

# There's lots of scope for doing really interesting reporting here. You do want the logs
from 8 different machines all displayed in the test reports, all in the best temporal order
that Lamport will allow.

# Long term, lots of scope for datamining the test results.

# Log analysis is tricky. You don't want tests that are brittle against log messages, leads
to many false failures.

# Performance testing is not something you can do on a virtual system, tricky w/ functional
tests. Benchmarking is a separate problem. You shoudn't be using your functional tests to
assess performance, as the functional tests are looking at the corner cases, trying to break
things, not simulate well-behaved code.

> Circus: Proposal and Preliminary Code for a Hadoop System Testing Framework
> ---------------------------------------------------------------------------
>
>                 Key: HADOOP-6248
>                 URL: https://issues.apache.org/jira/browse/HADOOP-6248
>             Project: Hadoop Common
>          Issue Type: New Feature
>          Components: test
>         Environment: Python, bash
>            Reporter: Alex Loddengaard
>         Attachments: HADOOP-6248.diff, HADOOP-6248_v2.diff, HADOOP-6248_v3.diff
>
>
> This issue contains a proposal and preliminary source code for Circus, a Hadoop system
testing framework.  At a high level, Circus will help Hadoop users and QA engineers to run
system tests on a configurable Hadoop cluster, or distribution of Hadoop.  See the comment
below for the proposal itself.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message