accumulo-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Josh Elser <>
Subject Re: Hadoop 2.0 Support for Accumulo 1.4 Branch
Date Tue, 12 Nov 2013 19:12:27 GMT
> Based on recent feedback on ACCUMULO-1792 and ACCUMULO-1795, I want to
> resurrect this thread to make sure everyone's concerns are addressed.
> For context, here's a link to the start of the last thread:
>  From ACCUMULO-1792, ctubbsii:
>> I'd be reluctant to support any Hadoop 2.x support in the 1.4 release
> line that breaks compatibility with 0.20. I don't think breaking 0.20
>> and then possibly fixing it again as a second step is acceptable (because
> that subsequent work may not ever be done, and I don't think
>> we should break the compatibility contract that we've established with
> 1.4.0).
> Chris, I believe keeping all of the work in a branch under the umbrella
> jira of ACCUMULO-1790 will ensure that we don't end up with a 1.4 release
> that doesn't have proper support for 0.20.203.
> Is there something beyond making sure the branch passes a full set of
> release tests on 0.20.203 that you'd like to see? In the event that the
> branch only ever contains the work for adding Hadoop 2, it's a simple
> matter to abandon without rolling into the 1.4 development line.
>  From ACCUMULO-1795, bills (and +1ed by elserj and ctubbsii):
>> I'm very uncomfortable with risking breaking continuity in such an old
> release, and I don't think managing two lines of 1.4 releases is
>> worth the effort. Though we have no official EOL policy, 1.3 was
> practically dead in the water once 1.4 was around, and I hope we start
>> encouraging more adoption of 1.5 (and soon 1.6) versus continually
> propping up 1.4.
> I'd love to get people to move off of 1.4. However, I think adding Hadoop 2
> support to 1.4 encourages this more than leaving it out.

I'm not sure I agree that adding Hadoop2 support to 1.4 encourages 
people to upgrade Accumulo. My gut reaction would be that it allows 
people to completely ignore Accumulo updates (ignoring moving to 1.4.5 
which would allow them to do hadoop2 with your proposed changes)

> Accumulo 1.5.x places a higher burden on HDFS than 1.4 did, and I'm not
> surprised people find relying on 0.20 for the 1.5 WAL intimidating.
> Upgrading both HDFS and Accumulo across major versions at once is asking
> them to take on a bunch of risk. By adding in Hadoop 2 support to 1.4 we
> allow them to break the risk up into steps: they can upgrade HDFS versions
> first, get comfortable, then upgrade Accumulo to 1.5.

Personally, maintaining 0.20 compatibility is not a big concern on my 
radar. If you're still running an 0.20 release, I'd *really* hope that 
you have an upgrade path to 1.2.x (if not 2.2.x) scheduled.

I think claiming that 1.5 has a higher burden on 1.4 is a bit of a 
fallacy. There were many problems and pains regarding WALs in <=1.4 that 
are very difficult to work with in a large environment (try finding WALs 
in server failure cases). I think the increased I/O on HDFS is a much 
smaller cost than the completely different I/O path that the old loggers 

I also think upgrading Accumulo is much less scary than upgrading HDFS, 
but that's just me.

To me, it seems like the argument may be coming down to whether or not 
we break 0.20 hadoop compatibility on a bug-fix release and how 
concerned we are about letting users lag behind the upstream development.

> I think the existing tickets under the umbrella of ACCUMULO-1790 should
> ensure that we end up with a single 1.4 line that can work with either the
> existing claimed in releases or against 2.2.0.
> Bill (or Josh or Chris), is there stronger language you'd like to see
> around docs / packaging (area #3 in the original plan and currently
> ACCUMULO-1796)? Maybe expressly only doing a binary convenience package for
> Are you looking for something beyond a full release suite to
> ensure 1.4 is still maintaining compatibility on Hadoop 0.20.203?

Again, my biggest concern here is not following our own guidelines of 
breaking changes across minor releases, but I'd hope 0.20 users have an 
upgrade path outlined for themselves.

View raw message