accumulo-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Christopher <>
Subject Re: [DISCUSS] More Versioning Discussion
Date Fri, 18 Apr 2014 15:42:45 GMT
On Fri, Apr 18, 2014 at 10:43 AM, Sean Busbey <> wrote:
> I know this got very long, especially for a Friday. Please bear with me.
> On Thu, Apr 17, 2014 at 5:56 PM, Christopher <> wrote:
>> You seem to keep insisting that we don't have consensus on basic API
>> guarantees. I don't think that's true. We may not have a complete
>> policy, but I think we have some agreement on some of the basics of
>> what we want users to be able to expect. It's still a good idea to
>> think about compatibility forwards and backwards, within a release
>> line, and I'm pretty sure we all agree on that. Lack of complete
>> policy is not the same as lack of agreement on some of the things that
>> policy would contain. Perhaps we've been too permissive in the past
>> and not pushed back as hard on it, in order to avoid controversy, but
>> I don't think it's a lack of agreement at play.
>> My question for the larger group is:
>> Am I wrong? Do we, or do we not, want compatibility between different
>> versions in a release line (1.4.x, 1.5.x, 1.6.x, etc.)?
> I'm sure that as a community we want backwards compatibility within a major
> version (currently the "y" in Version # x.y.z). I have heard you and Keith
> argue for forward compatibility within a major version and I have heard me
> argue against it. Even then, we have not really had a discussion about what
> we mean to cover by compatibility.
> We have an expressed scope of some packages in the Java API and a rough
> outline of major underlying services (Hadoop/ZooKeeper versions), but
> that's it. This is a very simplified view of compatibility. A
> non-exhaustive list of compatibility considerations:
> * the Thrift API amongst server components (though ACCUMULO-751 implies
> some)
> * the Thrift API between clients and servers (also implied in ACCUMULO-751)
> * the Thrift Proxy API
> * Data serialized to walogs
> * Data serialized to backing files in HDFS (though we have a convention
> that's come up a couple of times)
> * Data serialized to ZooKeeper
> * Data serialized in internal tables (metadata, root, trace)
> * visibility expressions (and labels themselves)
> * iterators
> * The Accumulo Launcher scripts ( / / the rest of bin)
> * The Accumulo Shell
> * Configuration files & defaults
> * Misc utility executables
> * The setup process
> * provided classpath
> * packaged tests
> * packaged examples
> * Log contents
> Maintaining a compatibility statement across the lot of these is, frankly,
> going to be hard. We already have problems just keeping binary backwards
> compatibility and that's the one where it's easiest to leverage external
> tooling. We should make sure we've had a good discussion and have good docs
> on the boundaries before we change what we're going to expect everyone to
> abide by.
>> In my view, *any* comprehensive versioning policy we adopt is going to
>> include the idea that the last segment of the versioning denotes a
>> bugfix release. Is there any possibility at all that we'd adopt a
>> policy that doesn't include this? I think not. So, why not be more
>> strict about this now?
> I agree that whatever versioning policy we adopt post-1.6 will treat the z
> in version # x.y.z as bugfix. I also expect we will have a fairly strict
> compatibility restriction on it. Part of why we'll be able to do that is
> that having a minor version will allow developers get most new feature
> ideas out the door without waiting for the long release cycle of a major
> version.
> These are our release cycles to date since coming to ASF (approx based on
> user@, jira, RC votes):
> 1.3.5 : 2011-12-17
> 1.3.6 : 2012-06-13
> 1.4.0 : 2012-04-03
> 1.4.1 : 2012-07-03
> 1.4.2 : 2012-11-15
> 1.4.3 : 2013-03-17
> 1.4.4 : 2013-08-22
> 1.4.5 : 2014-04-07
> 1.5.0 : 2013-05-23
> 1.5.1 : 2014-03-06
> 1.6.0 : 2014-04-21 (expected)
> That's ~1 year between major releases. IMO, that'd be a good rate presuming
> a maintenance window of ~2 major versions (I'd even like it to be a little
> slower). It's also somewhere between 3-9 months  between minor/bugfix
> versions. IMO, this is too slow for bugfix and okay for minor.
> It's true that adopting the restriction you suggest will help us speed up
> our rate of bugfix releases in the short term. For one, we could adopt a
> less rigorous release testing process. That would let us set a regular clip
> for pushing out whatever versions had fixes, say the first Wednesday of
> each month.
> However, there would be two detrimental affects. First, there's the
> frustration of additional delay for developers who want to see new features
> make it out. More importantly, forcing all feature additions to major
> versions couples them to the cost of non-backwards compatible API changes,
> which in turn discourages uptake with users.

We don't have to couple minor feature additions to major versions. We
could easily continue the 1.x line with minor feature releases (1.7,
1.8, etc.) with backwards compatibility guarantees to 1.5.0 (since
1.6.0 is already backwards compatible with 1.5.0, and we can't do
anything about earlier than that, but hopefully 1.4 can be EOL'd

> Once we have clear compatibility guidelines and the ability to distinguish
> between major, minor, and bugfix, we should get better at hitting a regular
> release stride. When we have a more regular stride, people will generally
> be more comfortable waiting for their particular contribution to go into a
> later release. Furthermore, having a steady stream of non-bugfix minor
> upgrades that don't require client changes will help improve our current
> situation wrt getting more recent versions into production.
> Until we have a formalized compatibility guide, I'd encourage us to allow
> our volunteers to target whatever version works for them so long as

My main concern with this last point is that it dramatically increases
the burden on the entire community (in terms of review and testing)
for what should be easy bugfix releases for supported branches. It
also significantly increases the risk of destabilization for long-term
support. While I think we're in full agreement for the future (post
1.6), I see no reason why we can't start adopting some of the
strictness today for the older versions the community is still
supporting, to immediately start gaining some of the benefits of this
reduced burden (say, for 1.4.6, 1.5.2, and 1.6.1).

It seems we generally agree on what *should* be done in the future,
but you're reluctant to adopt it gradually, instead creating a barrier
at some version, V, after which we apply these standards, and before
which (almost) anything goes. And, I think your reluctance is based
partially on the concern for precedence (which I don't think is
valid), and a concern for supporting minor features releases in 1.4.x
and 1.5.x (which I think is valid, but I don't think we should burden
ourselves with, since 1.4.x can be EOL'd soon and 1.6.0 can be
roughly* viewed as a minor backwards-compatible release to 1.5.0).

* Note: I say "roughly", because 1.6.0 did remove methods deprecated
in 1.5.0, and is only backwards-compatible for non-deprecated stuffs.
Obviously, we can be more strict about this in future 1.x releases.

> 1) we don't violate backwards compatibility
> 2) we handle stability requests with compromise positions
> A good example of the latter is ACCUMULO-1395, where the example
> configuration scripts were left in place since many users rely on them for
> bootstrapping.
> --
> Sean

View raw message