accumulo-notifications mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Josh Elser (JIRA)" <>
Subject [jira] [Commented] (ACCUMULO-3871) run integration tests as a map/reduce job
Date Thu, 04 Jun 2015 15:39:39 GMT


Josh Elser commented on ACCUMULO-3871:

bq. some of these tests have a lot of transitive dependencies (I'm looking at you, Kerberos).

:smile:. Yeah, I know that Apache Directory pulls in a ton of deps just to get the MiniKdc
functionality. It would be nice if we could work with them to pare that down in the future.

So, the suggested way of fixing this would be to move the test/src/test/java code into it's
own project, and move the classes from test/src/test/java to project/src/main/java.

The fact that these two jars share the org.apache.accumulo.test namespace has already been
an issue, so I had to stop Sealing the test-jar.

So, I would like to move these tests to a new project (it-test? test-it? cluster-test?), and
maybe shade in the dependencies so invoking the map-reduce job is simpler. I've never used
shading before, so I'm not sure if this would be prohibitive. Of course, the test jar would
not be in the accumulo-bin.tar.gz file unless we use a specific profile.

At first read, it seems like if we could make a mapreduce-it module (or something) that just
shades in the dependencies and test classes you need. Maybe you could keep the existing accumulo-test
jar off of the classpath and that would prevent you from having to unseal the existing accumulo-test
jar. Not entirely sure... this is the time I usually would ask [~ctubbsii] :P

> run integration tests as a map/reduce job
> -----------------------------------------
>                 Key: ACCUMULO-3871
>                 URL:
>             Project: Accumulo
>          Issue Type: Improvement
>          Components: test
>            Reporter: Eric Newton
>            Assignee: Eric Newton
>             Fix For: 1.8.0
>          Time Spent: 50m
>  Remaining Estimate: 0h
> When the functional tests were moved to java, we lost the ability to run the tests via
map/reduce.  It would be nice to run the ITs in under 2 hours. and take advantage of an entire
cluster, especially after making large sweeping changes.

This message was sent by Atlassian JIRA

View raw message