hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Kay Kay <kaykay.uni...@gmail.com>
Subject Re: Moving to Maven (HBASE-2099)
Date Sat, 13 Feb 2010 09:44:40 GMT
On 02/13/2010 01:29 AM, Dhruba Borthakur wrote:
>  From what I understand, the slowness of 'ivy' can be reduced if you can
> fetch dependent jars from local ivy server, isn't it?
The problem discussed is an artifact of hbase , trying to keep up with 
the most recent snapshots of hadoop-core/ hdfs / mapred and hence the 
ivy resolution is expensive that every time it hits the mvn repository 
to check for the latest snapshot , if any.  So the slowness is due to 
the necessity to keep up with the dependencies to identify issues early 
in the cycle. Specifically this can be attributed to the - 
changing="true" in all the ivy.xml-s in hbase, for hadoop artifacts . I 
am looking to making it a configurable option to avoid expensive build 

This will not be an issue if this were a hbase release, depending on 
other releases of hadoop-core / mapred / common etc.
Both ivy and maven does cache the artifacts locally making the roundtrip 
redundant (except for the first time, of course), so this should not be 
an issue for people trying to build the release from sources, since it 
should be moot by then.

> thanks,
> dhruba
> On Sat, Feb 13, 2010 at 12:25 AM, Kay Kay<kaykay.unique@gmail.com>  wrote:
>> Mathias -
>>    I have been using Ivy / Maven , interchangeably in different projects for
>> the build management.  Both of them clearly have their strong points and
>> drawbacks.  Ivy fits great for thrift because of the nature of tasks ,
>> involved using some external command-line (thrift generators) etc.  As I
>> mentioned before - HBase does not have such cross maven goals / between the
>> hairs as the build lifecycle is pretty straight-forward.
>>   In any case - the intention is to get to publish HBase artifacts and
>> maintain a smaller core and encouraging contribs. from the artifacts as
>> opposed to getting into the codebase.
>> Once there are HBase artifacts published , the contrib / plugins for the
>> same would be free to use ivy (with m2compatible="true") / maven as
>> appropriate.
>> Ryan -
>>    The slowness is attributed to the 'changing="true" ' in ivy.xml-s for all
>> the hadoop-common / -hdfs / -mapreduce snapshots that we are using. I am
>> facing similar 'slowness' with other mvn hadoop (snapshot) dependencies as
>> well. In retrospective, that should have been made a configurable flag in
>> libraries.properties , to ease things. Hopefully that is sorted out soon.
>> On 02/13/2010 12:10 AM, Ryan Rawson wrote:
>>> Would you mind elaborating more?  At the moment, most people do not
>>> build hbase, and the POM/jar/publishing thing is orthogonal - those
>>> who wish to build their own projects with ivy and/or ant are free to
>>> do so and not be impacted by our use of maven.
>>> We have ivy, but it doesnt integrate with our IDEs and is rather slow
>>> to build and rebuild.
>>> On Sat, Feb 13, 2010 at 12:03 AM, Mathias Herberts
>>> <mathias.herberts@gmail.com>   wrote:
>>>> -1
>>>> I think Maven is too complex and will lower the adoption of HBase by
>>>> people today willing to build it.
>>>> I would suggest using Ivy for dependency management as was done in
>>>> Thrift.
>>>> Mathias.

View raw message