hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Andrew Purtell <andrew.purt...@gmail.com>
Subject Re: Upgrading Hadoop dependencies and catching potential incompatibilities for HBase
Date Wed, 28 Sep 2016 20:16:33 GMT
Great suggestion!

> On Sep 28, 2016, at 11:55 AM, Enis Söztutar <enis.soz@gmail.com> wrote:
> 
> Can Hadoop please shade ALL of the dependencies (including PB) in Hadoop-3
> so that we do not have this mess going forward.
> 
> Enis
> 
>> On Wed, Sep 28, 2016 at 11:19 AM, Jonathan Hsieh <jon@cloudera.com> wrote:
>> 
>> You can build hbase using the profile by adding this -D setting.
>> 
>> mvn clean test install -Dhadoop.profile=3.0
>> 
>> At the moment the hbase's hadoop 3 profile has fails at mvn compile,
>> install, and test phases. There are some issues that are patch available
>> addressing these in HBase currently.
>> 
>> compile https://issues.apache.org/jira/browse/HBASE-16711
>> test https://issues.apache.org/jira/browse/HBASE-6581
>> install https://issues.apache.org/jira/browse/HBASE-16712
>> 
>> IIRC, currently hbase's hadoop3 profile currently tracks the hadoop-3.0
>> SNAPSHOT artifacts.  Sean, would these dependencies changes be targeted for
>> this branch and land in the next hadoop 3 alpha/beta?
>> Jon.
>> 
>> On Wed, Sep 28, 2016 at 9:56 AM, Sean Mackrory <mackrorysd@gmail.com>
>> wrote:
>> 
>>> HBase folks,
>>> 
>>> I'd like to work on upgrading some of the dependencies in Hadoop that are
>>> starting to lag behind:
>>> 
>>> jackson2 (https://issues.apache.org/jira/browse/HADOOP-12705)
>>> jaxb-api (https://issues.apache.org/jira/browse/HADOOP-13659)
>>> commons-configuration (https://issues.apache.org/
>> jira/browse/HADOOP-13660)
>>> htrace (https://issues.apache.org/jira/browse/HADOOP-13661)
>>> 
>>> Some of these (chiefly jackson) are known to be high risk upgrades and
>> have
>>> caused problems for HBase in the past. I wanted to give you a heads up
>>> about this work and coordinate with you to make sure I do all the testing
>>> you think is necessary to discover any potential problems well in advance
>>> of the breakages.
>>> 
>>> I was going to look at just running HBase's unit tests when built against
>>> my Hadoop's trunk with my changes (Should I expect any problems here? Is
>>> there a branch already focused on Hadoop 3 compatibility or anything?)
>> and
>>> Bigtop's integration tests too. Anything else you would add to this?
>>> 
>> 
>> 
>> 
>> --
>> // Jonathan Hsieh (shay)
>> // HBase Tech Lead, Software Engineer, Cloudera
>> // jon@cloudera.com // @jmhsieh
>> 

Mime
View raw message