accumulo-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Josh Elser <>
Subject Re: Reduced testing burden for bug-fix releases
Date Thu, 19 Jun 2014 19:42:13 GMT
For context, I do not have the physical resources available to me at any 
point in time.

Time is also a concern, but usually less of one because I don't mind 
working into evenings and weekends to get this done, and I can usually 
work on my other priorities concurrently.

On 6/19/14, 12:38 PM, Christopher wrote:
> I don't know Josh's concerns, but the concern for me is both resources and
> time. No matter how much resources we have, it is still not infinite, and
> I'd rather we focus our testing efforts on the changeset between the
> previous release and the minor/bugfix release, rather than spend resources
> and time on all the exhaustive general testing, which mostly exercises code
> that has not changed.
> For instance, do we really need 72-hours of continuous ingest on a large
> cluster to release a bugfix which affects the shell?
> If the long running tests are what is necessary to exercise the changeset,
> that makes sense, but otherwise, no.
> --
> Christopher L Tubbs II
> On Thu, Jun 19, 2014 at 3:20 PM, David Medinets <>
> wrote:
>> -1 I hesitate to step into this discussion because I can't also step
>> up and do the long-term testing even as I recommend that it must be
>> done. There are at least four companies supporting Accumulo and
>> contributing back to the project. Surely one of those companies can
>> supply the resources to continue the existing test regimen? Is there
>> some concern that those resources won't be available for the next
>> release cycle?
>> On Thu, Jun 19, 2014 at 3:13 PM, Josh Elser <> wrote:
>>> As we're starting to consider 1.5.2 and 1.6.1 coming out in the near
>> future,
>>> I want to revisit a discussion[1] I started at the end of April regarding
>>> the "testing burden" that is currently set forth in our release
>> document[2].
>>> What I'm proposing is to modify the language of the release document to
>> be
>>> explicit about the amount of testing needed. For bug-fix, "minor"
>> releases
>>> (e.g. 1.5.2 and 1.6.1), the 7 days of testing using continuous ingest and
>>> randomwalk (with and without agitation) will be clearly defined as "may"
>>> instead of "should" or "must" language. If the resources are available,
>> it
>>> is recommended that some longer, multi-process/node test is run against
>> the
>>> release candidate; however, it is not required and should not prevent us
>>> from making the minor release.
>>> I will also include language that strongly recommends that the changes
>>> included in the "minor" release be vetted/reviewed as a way to mitigate
>> the
>>> risk of shipping new regressions.
>>> I am not recommending that the language be changed for "major" releases
>>> (e.g. 1.7.0 and 2.0.0) as these releases still imply significant new
>>> features or internal changes.
>>> Unless someone informs me otherwise, I will treat this as a normal
>>> lazy-consensus approval. Assuming we move closer to "proper" semantic
>>> versioning for 2.0.0, I believe these updated guidelines will change
>> again.
>>> I do however think there is merit in making this change now so that we
>> can
>>> get the good bugs that we've fixed out to our users.
>>> Let me know what you think. I will wait, at least, the prescribed three
>> days
>>> before changing any thing.
>>> - Josh
>>> [1]
>>> [2]

View raw message