accumulo-notifications mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Josh Elser (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (ACCUMULO-4423) Define categories for integration test execution
Date Wed, 31 Aug 2016 04:46:21 GMT

    [ https://issues.apache.org/jira/browse/ACCUMULO-4423?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15451108#comment-15451108
] 

Josh Elser commented on ACCUMULO-4423:
--------------------------------------

Alright, I pushed the "result" from the first round of reviews. Let's keep this open to address
any further concerns (it's so difficult to manage all of these changes when a full test of
every little thing in on the order of hours):

One point was the worry of uncategorized tests that would be ignored by an invocation of {{mvn
verify}}. The only one I see of these is {{./maven-plugin/src/it/plugin-test/src/test/java/org/apache/accumulo/plugin/it/PluginIT.java}}
which I think would still be getting run all the time as its driven by the invoker-plugin
instead. This needs to be verified. ACCUMULO-4432 can track most of this, I believe.

Another point of concern was the profiles introduce to easily enable groups of these properties.
They change

{noformat}
mvn verify -Daccumulo.it.groups=org.apache.accumulo.test.categories.AnyClusterTest -Daccumulo.it.excludeGroups=org.apache.accumulo.test.categories.MiniClusterOnlyTests
{noformat}

to 

{noformat}
mvn verify -Pstandalone-capable-tests
{noformat}

and

{noformat}
mvn verify -Daccumulo.it.groups=org.apache.accumulo.test.categories.MiniClusterOnlyTests -Daccumulo.it.excludedGroups=org.apache.accumulo.test.categories.AnyClusterTests
{noformat}

to

{noformat}
mvn verify -Pminicluster-only-tests
{noformat}

I'm not sure how to have a technical debate when the concern is over hypothetical groups/test-dimensions
that are added in the future. This needs to be hashed out some more apparently, because I
think the above are great. Personally, I equate it to what HBase have. They have profiles
for small, medium, and large tests. They also have profiles for devTesting (small+medium)
or allTests (small+medium+large). Their setup is a bit different than what we now have, but
i think the general plus-side is the same.

Documentation can likely always be improved. Other devs weighing in here would be good (not
just Christopher and myself arguing).

Finally, this will just need a close eye to make sure nothing else is broken. Best as I can
tell across three branches with 4 different configurations, everything is ok, but, knowing
my luck, something is busted.



> Define categories for integration test execution
> ------------------------------------------------
>
>                 Key: ACCUMULO-4423
>                 URL: https://issues.apache.org/jira/browse/ACCUMULO-4423
>             Project: Accumulo
>          Issue Type: Improvement
>          Components: build, test
>            Reporter: Josh Elser
>            Assignee: Josh Elser
>             Fix For: 1.7.3, 1.8.1, 2.0.0
>
>          Time Spent: 8h 40m
>  Remaining Estimate: 0h
>
> Had a thought the other day when talking to some folks about testing.
> Our current integration tests can be broken down into two categories:
> * Encapsulated (MiniCluster only)
> * Capable of using a Standalone Accumulo instance (instead of a MiniCluster)
> with any ITs that don't have MiniClusters (if there are any), I am lumping them into
the Encapsulated category.
> Now, if I have an environment set up which I can run these Standalone tests against,
it would be nice to just be able to execute those tests. I could then run the unit tests,
checkstyle, findbugs, and encapsulated integration tests out of band, not tied to the lifetime
(and cost) of that environment for the Standalone tests.
> It would be nice if we can use something like JUnit categories to group these tests and
then provide the "hooks" in the build to let users specify which categories are executed.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message