hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Dan Washusen <...@reactive.org>
Subject Re: Moving to Maven (HBASE-2099)
Date Sat, 13 Feb 2010 10:11:13 GMT
Maven provides a "-o" argument and Ivy provides a "-Doffline=true"
property.  I haven't used the Ivy variant but if you're using SNAPSHOT
dependencies it makes the build a lot quicker.  I usually do an infrequent
online build then do the rest in offline mode...

Maven also provides a "snapshot policy" that can be used to define the
interval between snapshot checks.  For example, "always", "daily",
"interval:<in minutes>" or "never"...
http://maven.apache.org/ref/2.0.8/maven-settings/settings.html#class_snapshots

Cheers,
Dan

On 13 February 2010 20:51, Dhruba Borthakur <dhruba@gmail.com> wrote:

> My personal experience is that the ivy-maven-stuff introduced into the
> Hadoop build system has tremendously slowed down the Hadoop build process.
> I
> am sure that this disadvantage is offet-ed by some advantages that I am not
> aware of. If you could educate me on the top two advantages that accrued to
> Hadoop after moving to the new build process, that would be awesome.
>
> thanks a bunch,
> dhruba
>
>
> On Sat, Feb 13, 2010 at 1:44 AM, Kay Kay <kaykay.unique@gmail.com> wrote:
>
> > On 02/13/2010 01:29 AM, Dhruba Borthakur wrote:
> >
> >>  From what I understand, the slowness of 'ivy' can be reduced if you can
> >> fetch dependent jars from local ivy server, isn't it?
> >>
> >>
> > The problem discussed is an artifact of hbase , trying to keep up with
> the
> > most recent snapshots of hadoop-core/ hdfs / mapred and hence the ivy
> > resolution is expensive that every time it hits the mvn repository to
> check
> > for the latest snapshot , if any.  So the slowness is due to the
> necessity
> > to keep up with the dependencies to identify issues early in the cycle.
> > Specifically this can be attributed to the - changing="true" in all the
> > ivy.xml-s in hbase, for hadoop artifacts . I am looking to making it a
> > configurable option to avoid expensive build time.
> >
> >
> > This will not be an issue if this were a hbase release, depending on
> other
> > releases of hadoop-core / mapred / common etc.
> > Both ivy and maven does cache the artifacts locally making the roundtrip
> > redundant (except for the first time, of course), so this should not be
> an
> > issue for people trying to build the release from sources, since it
> should
> > be moot by then.
> >
> >
> >
> >
> >
> >  thanks,
> >> dhruba
> >>
> >> On Sat, Feb 13, 2010 at 12:25 AM, Kay Kay<kaykay.unique@gmail.com>
> >>  wrote:
> >>
> >>
> >>
> >>> Mathias -
> >>>   I have been using Ivy / Maven , interchangeably in different projects
> >>> for
> >>> the build management.  Both of them clearly have their strong points
> and
> >>> drawbacks.  Ivy fits great for thrift because of the nature of tasks ,
> >>> involved using some external command-line (thrift generators) etc.  As
> I
> >>> mentioned before - HBase does not have such cross maven goals / between
> >>> the
> >>> hairs as the build lifecycle is pretty straight-forward.
> >>>  In any case - the intention is to get to publish HBase artifacts and
> >>> maintain a smaller core and encouraging contribs. from the artifacts as
> >>> opposed to getting into the codebase.
> >>> Once there are HBase artifacts published , the contrib / plugins for
> the
> >>> same would be free to use ivy (with m2compatible="true") / maven as
> >>> appropriate.
> >>>
> >>> Ryan -
> >>>   The slowness is attributed to the 'changing="true" ' in ivy.xml-s for
> >>> all
> >>> the hadoop-common / -hdfs / -mapreduce snapshots that we are using. I
> am
> >>> facing similar 'slowness' with other mvn hadoop (snapshot) dependencies
> >>> as
> >>> well. In retrospective, that should have been made a configurable flag
> in
> >>> libraries.properties , to ease things. Hopefully that is sorted out
> soon.
> >>>
> >>>
> >>>
> >>>
> >>> On 02/13/2010 12:10 AM, Ryan Rawson wrote:
> >>>
> >>>
> >>>
> >>>> Would you mind elaborating more?  At the moment, most people do not
> >>>> build hbase, and the POM/jar/publishing thing is orthogonal - those
> >>>> who wish to build their own projects with ivy and/or ant are free to
> >>>> do so and not be impacted by our use of maven.
> >>>>
> >>>> We have ivy, but it doesnt integrate with our IDEs and is rather slow
> >>>> to build and rebuild.
> >>>>
> >>>> On Sat, Feb 13, 2010 at 12:03 AM, Mathias Herberts
> >>>> <mathias.herberts@gmail.com>   wrote:
> >>>>
> >>>>
> >>>>
> >>>>
> >>>>> -1
> >>>>>
> >>>>> I think Maven is too complex and will lower the adoption of HBase
by
> >>>>> people today willing to build it.
> >>>>>
> >>>>> I would suggest using Ivy for dependency management as was done
in
> >>>>> Thrift.
> >>>>>
> >>>>> Mathias.
> >>>>>
> >>>>>
> >>>>>
> >>>>>
> >>>>>
> >>>>
> >>>>
> >>>
> >>>
> >>
> >>
> >>
> >
> >
>
>
> --
> Connect to me at http://www.facebook.com/dhruba
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message