hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Stack <st...@duboce.net>
Subject Re: scoping integration tests
Date Wed, 30 Nov 2011 18:11:34 GMT
On Tue, Nov 29, 2011 at 10:28 PM, Jesse Yates <jesse.k.yates@gmail.com> wrote:
> I don't know if we should consider the integration tests as the primary
> thing that bigtop runs. Since they are deploying to a small cluster
> (right?), the tests that FB is supposed to be open sourcing soon should
> probably be what they run (and lets call those acceptance tests).
> While integration tests could be run by bigtop, they should definitely be
> part of our testing of patches as well - no patches should go into trunk
> that don't pass both the unit tests and integration tests. Then we push in
> bigger patches or cut releases we make sure that the acceptance tests pass.

Chatting w/ Roman and Mikhail yesterday at the Hackathon, and with a
probably broader goals than perhaps what you've been thinking on
Jesse, I think:

+ IntegrationTests* -- where IntegrationTest* is tests with the
IntegrationTest prefix and are that run when  'mvn verify' as per your
recent contrib Jesse -- differ from unit tests in that they always run
against a cluster
+ IntegrationTests* do not expect to be able to put their fingers into
regionservers and/or master to figure test/pass or fail; they treat
the cluster as 'remote'
+ IntegrationTest* do not concern themselves with cluster setup and
teardown -- this is done external to the test code by the failsafe
plugin + configs (If we do this and the previous point, then our
IntegrationTest* can be run by bigtop if I understood Roman properly
+ Cluster setup and teardown needs to be pluggable (it seems like
failsafe can do this for us).  We need this because default cluster
will be minicluster but then it needs to be easy for us to plug in a
whirr cluster and/or for the likes of our fb homies to plugin their
cluster setup, etc.
+ IntegerationTest* are sizeable. You can pass things like do 1M rows
or 100B rows -- and configurable ("Use 'that' cluster")

I don't think a patch needs to pass all integration tests before it
gets committed.  If a bit of functionality needs an integration-type
test, then IMO it shoudl be cast as a 'large' unit test run by
surefire.  As per N, devs do 'mvn test' which will run small and
medium unit tests and then our CI does small/medium/large.

I don't want to get hung up on nomenclature.  If there are long email
threads on test taxonomy, then something has gone awry IMO.  A clean
surefire for unit tests and failsafe for integration tests where unit
tests are whats runnable w/o need of outside dependencies -- e.g. an
idling hbase or mr cluster -- and is what devs and our CI does seems
like a pretty clean separation to me, one that even us hbase devs
might grok.

What do you reckon?


View raw message