accumulo-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Sean Busbey <bus...@cloudera.com>
Subject Re: [DISCUSS] More Versioning Discussion
Date Fri, 18 Apr 2014 14:43:55 GMT
I know this got very long, especially for a Friday. Please bear with me.


On Thu, Apr 17, 2014 at 5:56 PM, Christopher <ctubbsii@apache.org> wrote:


> You seem to keep insisting that we don't have consensus on basic API
>
> guarantees. I don't think that's true. We may not have a complete
> policy, but I think we have some agreement on some of the basics of
> what we want users to be able to expect. It's still a good idea to
> think about compatibility forwards and backwards, within a release
> line, and I'm pretty sure we all agree on that. Lack of complete
> policy is not the same as lack of agreement on some of the things that
> policy would contain. Perhaps we've been too permissive in the past
> and not pushed back as hard on it, in order to avoid controversy, but
> I don't think it's a lack of agreement at play.
>
> My question for the larger group is:
>
> Am I wrong? Do we, or do we not, want compatibility between different
> versions in a release line (1.4.x, 1.5.x, 1.6.x, etc.)?
>
>
I'm sure that as a community we want backwards compatibility within a major
version (currently the "y" in Version # x.y.z). I have heard you and Keith
argue for forward compatibility within a major version and I have heard me
argue against it. Even then, we have not really had a discussion about what
we mean to cover by compatibility.

We have an expressed scope of some packages in the Java API and a rough
outline of major underlying services (Hadoop/ZooKeeper versions), but
that's it. This is a very simplified view of compatibility. A
non-exhaustive list of compatibility considerations:

* the Thrift API amongst server components (though ACCUMULO-751 implies
some)
* the Thrift API between clients and servers (also implied in ACCUMULO-751)
* the Thrift Proxy API
* Data serialized to walogs
* Data serialized to backing files in HDFS (though we have a convention
that's come up a couple of times)
* Data serialized to ZooKeeper
* Data serialized in internal tables (metadata, root, trace)
* visibility expressions (and labels themselves)
* iterators
* The Accumulo Launcher scripts (accumulo.sh / tool.sh / the rest of bin)
* The Accumulo Shell
* Configuration files & defaults
* Misc utility executables
* The setup process
* provided classpath
* packaged tests
* packaged examples
* Log contents

Maintaining a compatibility statement across the lot of these is, frankly,
going to be hard. We already have problems just keeping binary backwards
compatibility and that's the one where it's easiest to leverage external
tooling. We should make sure we've had a good discussion and have good docs
on the boundaries before we change what we're going to expect everyone to
abide by.


> In my view, *any* comprehensive versioning policy we adopt is going to
> include the idea that the last segment of the versioning denotes a
> bugfix release. Is there any possibility at all that we'd adopt a
> policy that doesn't include this? I think not. So, why not be more
> strict about this now?
>
>
I agree that whatever versioning policy we adopt post-1.6 will treat the z
in version # x.y.z as bugfix. I also expect we will have a fairly strict
compatibility restriction on it. Part of why we'll be able to do that is
that having a minor version will allow developers get most new feature
ideas out the door without waiting for the long release cycle of a major
version.

These are our release cycles to date since coming to ASF (approx based on
user@, jira, RC votes):

1.3.5 : 2011-12-17
1.3.6 : 2012-06-13

1.4.0 : 2012-04-03
1.4.1 : 2012-07-03
1.4.2 : 2012-11-15
1.4.3 : 2013-03-17
1.4.4 : 2013-08-22
1.4.5 : 2014-04-07

1.5.0 : 2013-05-23
1.5.1 : 2014-03-06

1.6.0 : 2014-04-21 (expected)

That's ~1 year between major releases. IMO, that'd be a good rate presuming
a maintenance window of ~2 major versions (I'd even like it to be a little
slower). It's also somewhere between 3-9 months  between minor/bugfix
versions. IMO, this is too slow for bugfix and okay for minor.

It's true that adopting the restriction you suggest will help us speed up
our rate of bugfix releases in the short term. For one, we could adopt a
less rigorous release testing process. That would let us set a regular clip
for pushing out whatever versions had fixes, say the first Wednesday of
each month.

However, there would be two detrimental affects. First, there's the
frustration of additional delay for developers who want to see new features
make it out. More importantly, forcing all feature additions to major
versions couples them to the cost of non-backwards compatible API changes,
which in turn discourages uptake with users.

Once we have clear compatibility guidelines and the ability to distinguish
between major, minor, and bugfix, we should get better at hitting a regular
release stride. When we have a more regular stride, people will generally
be more comfortable waiting for their particular contribution to go into a
later release. Furthermore, having a steady stream of non-bugfix minor
upgrades that don't require client changes will help improve our current
situation wrt getting more recent versions into production.

Until we have a formalized compatibility guide, I'd encourage us to allow
our volunteers to target whatever version works for them so long as

1) we don't violate backwards compatibility

2) we handle stability requests with compromise positions

A good example of the latter is ACCUMULO-1395, where the example
configuration scripts were left in place since many users rely on them for
bootstrapping.

-- 
Sean

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message