accumulo-notifications mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Josh Elser (JIRA)" <>
Subject [jira] [Commented] (ACCUMULO-3167) Decouple MiniAccumuloCluster from integration test base class
Date Wed, 24 Sep 2014 23:23:35 GMT


Josh Elser commented on ACCUMULO-3167:

My initial thoughts on how to implement this is to introduce an extra shim between integration
tests and the underlying cluster implementation (e.g. SimpleMacIT). Thus, our ITs that need
a running instance of Accumulo will extend this new class that provides a "deployment agnostic"
set of functions to interact with that instance.

Some complexity issues arise around this because some deployments (notably, the plain-jane
Accumulo instance) cannot guarantee the breadth of functionality that some tests expect: the
inability to restart Accumulo processes, control site configuration files (HDFS or Accumulo),
and alter JVM level properties. All other configuration *should* be capable of modification
via use of ZooKeeper through the normal API. MAC and Slider (Accumulo on YARN) both solve
these problems because they can dynamically start a new instance with the requested configuration
-- I've been referring to this as "managed" clusters.

My initial intent is to support running tests against MAC as currently happens (the default
won't change), provide some configuration to allow use an existing vanilla Accumulo instance,
ability to skip tests which require "managed" clusters, and, if not full support initially,
changes which will help Slider integration down the road.

> Decouple MiniAccumuloCluster from integration test base class
> -------------------------------------------------------------
>                 Key: ACCUMULO-3167
>                 URL:
>             Project: Accumulo
>          Issue Type: Improvement
>          Components: mini, test
>            Reporter: Josh Elser
>            Assignee: Josh Elser
>             Fix For: 1.6.1, 1.7.0
> (apologies if I already had a ticket for this somewhere, I couldn't find it)
> Our integration tests are very nice and automated at the moment because we can use MiniAccumuloCluster
to "provision" an Accumulo instance (or used a shared instance), and run a test against it.
For the most part, this works well and provides an accurate test harness.
> Thus, to run integration tests, you need a sufficiently beefy machine since the same
host will be running all of Accumulo as well as performing any client work. When resources
are available to use, it would be nice to leverage them -- whether these are yarn, mesos,
a vanila installation, etc.
> In addition to the additional computational power from using extra hardware, it also
encourages us to use the public API as much as possible instead of relying on "hidden" impl
methods in MiniAccumuloCluster.
> I propose making changes to the IT test base (AbstractMacIT, SimpleMacIT, ConfigurableMacIT)
to add an extra step between them an test classes to allow "injection" of a more generic Accumulo
"cluster" that is not associated with MAC.

This message was sent by Atlassian JIRA

View raw message