accumulo-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Sean Busbey <bus...@clouderagovt.com>
Subject Re: Hadoop 2.0 Support for Accumulo 1.4 Branch
Date Tue, 12 Nov 2013 14:22:53 GMT
On Fri, Oct 18, 2013 at 12:29 AM, Sean Busbey <busbey@cloudera.com> wrote:

> On Tue, Oct 15, 2013 at 10:20 AM, Sean Busbey <busbey@cloudera.com> wrote:
>
>>
>> On Tue, Oct 15, 2013 at 10:16 AM, Sean Busbey <busbey@cloudera.com>wrote:
>>
>>>
>>> On Tue, Oct 15, 2013 at 7:16 AM, <dlmarion@comcast.net> wrote:
>>>
>>>> Just to be clear, we are talking about adding profile support to the
>>>> pom's for Hadoop 2.2.0 for a 1.4.5 and 1.5.1 release, correct? We are not
>>>> talking about changing the default build profile for these branches are we?
>>>>
>>>>
>>>>
>>> for 1.4.5-SNAPSHOT I am only talking about adding support Hadoop 2.2.0.
>>> I am not suggesting we change the default from building against Hadoop
>>> 0.23.203.
>>>
>>>
>>>
>> I mean 0.20.203.0. Ugh, Hadoop versions.
>>
>>
>
> Okay, barring additional suggestions, tomorrow afternoon I'll break things
> down into an umbrella and 3 sub tasks:
>
> 1) addition of hadoop 2 support
>
>  - to include backports of commits
>  - to include making the target hadoop 2 version 2.2.0
>  - to include test changes that flex hadoop 2 features like fail over
>
> 2) ensuring compatibility for 0.20.203
>
> - presuming some subset of the commits in 1) will break it since 0.20
> support was left behind in 1.5
>
> 3) doc / packaging updates
>
> - the issue of binary releases per distro
> - doc patch for what version(s) the release tests are expected to run
> against
>
> Once work is put against those tickets, I'd expect things to go into a
> branch based on the umbrella ticket until such time as the complete work
> can pass the test suite that we'll use at the next release. Then it can get
> rebased onto the 1.4.x dev branch.
>
> --
> Sean
>

Based on recent feedback on ACCUMULO-1792 and ACCUMULO-1795, I want to
resurrect this thread to make sure everyone's concerns are addressed.

For context, here's a link to the start of the last thread:

http://bit.ly/1aPqKuH

>From ACCUMULO-1792, ctubbsii:

> I'd be reluctant to support any Hadoop 2.x support in the 1.4 release
line that breaks compatibility with 0.20. I don't think breaking 0.20
> and then possibly fixing it again as a second step is acceptable (because
that subsequent work may not ever be done, and I don't think
> we should break the compatibility contract that we've established with
1.4.0).

Chris, I believe keeping all of the work in a branch under the umbrella
jira of ACCUMULO-1790 will ensure that we don't end up with a 1.4 release
that doesn't have proper support for 0.20.203.

Is there something beyond making sure the branch passes a full set of
release tests on 0.20.203 that you'd like to see? In the event that the
branch only ever contains the work for adding Hadoop 2, it's a simple
matter to abandon without rolling into the 1.4 development line.

>From ACCUMULO-1795, bills (and +1ed by elserj and ctubbsii):

> I'm very uncomfortable with risking breaking continuity in such an old
release, and I don't think managing two lines of 1.4 releases is
> worth the effort. Though we have no official EOL policy, 1.3 was
practically dead in the water once 1.4 was around, and I hope we start
> encouraging more adoption of 1.5 (and soon 1.6) versus continually
propping up 1.4.

I'd love to get people to move off of 1.4. However, I think adding Hadoop 2
support to 1.4 encourages this more than leaving it out.

Accumulo 1.5.x places a higher burden on HDFS than 1.4 did, and I'm not
surprised people find relying on 0.20 for the 1.5 WAL intimidating.
Upgrading both HDFS and Accumulo across major versions at once is asking
them to take on a bunch of risk. By adding in Hadoop 2 support to 1.4 we
allow them to break the risk up into steps: they can upgrade HDFS versions
first, get comfortable, then upgrade Accumulo to 1.5.

I think the existing tickets under the umbrella of ACCUMULO-1790 should
ensure that we end up with a single 1.4 line that can work with either the
existing 0.20.203.0 claimed in releases or against 2.2.0.

Bill (or Josh or Chris), is there stronger language you'd like to see
around docs / packaging (area #3 in the original plan and currently
ACCUMULO-1796)? Maybe expressly only doing a binary convenience package for
0.20.203.0? Are you looking for something beyond a full release suite to
ensure 1.4 is still maintaining compatibility on Hadoop 0.20.203?


-Sean

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message