accumulo-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From John Vines <>
Subject Re: Releasing 1.5
Date Tue, 07 May 2013 15:10:58 GMT
I would also like to point out that hbase is putting out separate releases
for hadoop1 and hadoop2 ( They also have
support for both via maven, however they implemented a compatibility module
( which brings the schism
down to a single jar that needs to be interchanged. That may be something
we want to consider for 1.6.

The reason that I care about this is I'm working on things on top of
Accumulo, but against multiple versions of hadoop. I want to be able to
easily able to build against different versions of Accumulo 1.5 without
have to kill my local repo, reinstall accumulo built against my target
version of hadoop, etc. etc. It would be SOOOO much more convenient to just
switch my accumulo version from 1.5 to 1.5-hadoop2 and be done with it.

On Tue, Apr 30, 2013 at 12:32 AM, John Vines <> wrote:

> I've always been an advocate of sticking to vanilla compatibility, but
> maintaining ability to be compatible with other versions. Hadoop 2ish
> things are the first case where we are beginning to see broken run-time
> compatibility due to some API changes. While the fragmented state of hadoop
> creates a larger set of jars, even just hadoop 1 vs. hadoop2 is enough to
> break things. I think priority number 1 should be compile time
> compatibility with everything, followed by attempts for full runtime
> compatibility. Obviously this can't happen, but it can be achieved by
> identical source but split compiled resources, and I think that may be
> something we have to do. If we're putting in the legwork to know how to
> successfully run against hadoop_variant_8271, we may as well provide a
> compiled unit for it as well.
> On Tue, Apr 30, 2013 at 12:01 AM, Josh Elser <> wrote:
>> Funny enough, I gothit by these shenanigans last night when I was trying
>> to run trunk against CDH3 locally. After working through jars that were
>> marked asprovidedand weren't, and then running into
>> I threw in the towel and called it a night.
>> I think one thing we can all agree upon is that the "fragmented" state of
>> Hadoop distributions is a pain to work around; however, we do have a very
>> broad coverage across that variance just on our committer list. Considering
>> Benson's comments on the subject of "supporting" non-Apache Hadoop
>> variants, I would think that it's in our best interest to provide some
>> level of warm-fuzzy in terms of support. I'm worried about making people
>> chase their tails just to get Accumulo up and running on their flavor of
>> choice.
>> As far as what we distribute, I'm still of the mindset that support for
>> building Accumulo against other versions of Hadoop can be satisfied by
>> instructions on how to do so. Thus, I would say that Accumulo's default
>> dependency should continue to track Apache Hadoop's stable as it currently
>> does (maybe revisiting classifiers for 1.6?). I would say we can revisit
>> the subject of the src jars we publish when/if a flavor breaks Accumulo's
>> compilation.
>> Thoughts?
>> On 4/26/2013 4:35 PM, John Vines wrote:
>>> I had issues running a hadoop2 compiled version of accumulo against
>>> CDH4, I
>>> can't remember the specifics of it though.
>>> When I said specialized packaging, I was thinking of a naming convention
>>> to
>>> distinguish hadoop1 vs. hadoop2 ( vs. vendor-specific hadoop) compiled
>>> jars.
>>> On Fri, Apr 26, 2013 at 4:19 PM, Billie Rinaldi <
>>>  I'm not sure we are talking about actual vendor-specific code.  We are
>>>> deciding whether or not to create additional release tarballs that have
>>>> been compiled against various vendors' Hadoop-compatible file systems.
>>>> Assuming that we determine there is nothing prohibiting us from doing
>>>> this,
>>>> I think it would simply be up to the release manager (i.e. anyone who
>>>> assembles a release and calls a vote for it).  If someone cares enough
>>>> about a particular distribution to build and create an extra tarball,
>>>> they
>>>> can.  However, I don't think this is common for Apache projects --
>>>> additional packaging is usually left to supporting companies.  I haven't
>>>> even noticed any releases yet that come in Hadoop 1 and Hadoop 2
>>>> flavors.
>>>> I haven't heard (until now) that Accumulo compiled against an
>>>> appropriate
>>>> version of Apache Hadoop will not work with CDH, but John says that's
>>>> the
>>>> case.  John, have you tried this?  Also, what is the "specialized
>>>> packaging" you referred to?
>>>> On Fri, Apr 26, 2013 at 12:32 PM, David Medinets
>>>> <>**wrote:
>>>>  Does it make sense to put vendor-specific stuff under a
>>>>> contribs/vendors
>>>>> directory? Doing so would certainly indicate that we are
>>>>> vendor-agnostic.
>>>>> And give vendors an obvious place to contribute.

  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message