hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Andrew Purtell <apurt...@apache.org>
Subject Re: Moving to Maven (HBASE-2099)
Date Sat, 13 Feb 2010 17:36:46 GMT
-Doffline=true still wants to connect to resolve dependencies, so I think
it is a mis-named option.



----- Original Message ----
> From: Dan Washusen <dan@reactive.org>
> To: hbase-dev@hadoop.apache.org
> Sent: Sat, February 13, 2010 2:11:13 AM
> Subject: Re: Moving to Maven (HBASE-2099)
> 
> Maven provides a "-o" argument and Ivy provides a "-Doffline=true"
> property.  I haven't used the Ivy variant but if you're using SNAPSHOT
> dependencies it makes the build a lot quicker.  I usually do an infrequent
> online build then do the rest in offline mode...
> 
> Maven also provides a "snapshot policy" that can be used to define the
> interval between snapshot checks.  For example, "always", "daily",
> "interval:" or "never"...
> http://maven.apache.org/ref/2.0.8/maven-settings/settings.html#class_snapshots
> 
> Cheers,
> Dan
> 
> On 13 February 2010 20:51, Dhruba Borthakur wrote:
> 
> > My personal experience is that the ivy-maven-stuff introduced into the
> > Hadoop build system has tremendously slowed down the Hadoop build process.
> > I
> > am sure that this disadvantage is offet-ed by some advantages that I am not
> > aware of. If you could educate me on the top two advantages that accrued to
> > Hadoop after moving to the new build process, that would be awesome.
> >
> > thanks a bunch,
> > dhruba
> >
> >
> > On Sat, Feb 13, 2010 at 1:44 AM, Kay Kay wrote:
> >
> > > On 02/13/2010 01:29 AM, Dhruba Borthakur wrote:
> > >
> > >>  From what I understand, the slowness of 'ivy' can be reduced if you can
> > >> fetch dependent jars from local ivy server, isn't it?
> > >>
> > >>
> > > The problem discussed is an artifact of hbase , trying to keep up with
> > the
> > > most recent snapshots of hadoop-core/ hdfs / mapred and hence the ivy
> > > resolution is expensive that every time it hits the mvn repository to
> > check
> > > for the latest snapshot , if any.  So the slowness is due to the
> > necessity
> > > to keep up with the dependencies to identify issues early in the cycle.
> > > Specifically this can be attributed to the - changing="true" in all the
> > > ivy.xml-s in hbase, for hadoop artifacts . I am looking to making it a
> > > configurable option to avoid expensive build time.
> > >
> > >
> > > This will not be an issue if this were a hbase release, depending on
> > other
> > > releases of hadoop-core / mapred / common etc.
> > > Both ivy and maven does cache the artifacts locally making the roundtrip
> > > redundant (except for the first time, of course), so this should not be
> > an
> > > issue for people trying to build the release from sources, since it
> > should
> > > be moot by then.
> > >
> > >
> > >
> > >
> > >
> > >  thanks,
> > >> dhruba
> > >>
> > >> On Sat, Feb 13, 2010 at 12:25 AM, Kay Kay
> > >>  wrote:
> > >>
> > >>
> > >>
> > >>> Mathias -
> > >>>   I have been using Ivy / Maven , interchangeably in different projects
> > >>> for
> > >>> the build management.  Both of them clearly have their strong points
> > and
> > >>> drawbacks.  Ivy fits great for thrift because of the nature of tasks
,
> > >>> involved using some external command-line (thrift generators) etc.
 As
> > I
> > >>> mentioned before - HBase does not have such cross maven goals / between
> > >>> the
> > >>> hairs as the build lifecycle is pretty straight-forward.
> > >>>  In any case - the intention is to get to publish HBase artifacts and
> > >>> maintain a smaller core and encouraging contribs. from the artifacts
as
> > >>> opposed to getting into the codebase.
> > >>> Once there are HBase artifacts published , the contrib / plugins for
> > the
> > >>> same would be free to use ivy (with m2compatible="true") / maven as
> > >>> appropriate.
> > >>>
> > >>> Ryan -
> > >>>   The slowness is attributed to the 'changing="true" ' in ivy.xml-s
for
> > >>> all
> > >>> the hadoop-common / -hdfs / -mapreduce snapshots that we are using.
I
> > am
> > >>> facing similar 'slowness' with other mvn hadoop (snapshot) dependencies
> > >>> as
> > >>> well. In retrospective, that should have been made a configurable flag
> > in
> > >>> libraries.properties , to ease things. Hopefully that is sorted out
> > soon.
> > >>>
> > >>>
> > >>>
> > >>>
> > >>> On 02/13/2010 12:10 AM, Ryan Rawson wrote:
> > >>>
> > >>>
> > >>>
> > >>>> Would you mind elaborating more?  At the moment, most people do
not
> > >>>> build hbase, and the POM/jar/publishing thing is orthogonal - those
> > >>>> who wish to build their own projects with ivy and/or ant are free
to
> > >>>> do so and not be impacted by our use of maven.
> > >>>>
> > >>>> We have ivy, but it doesnt integrate with our IDEs and is rather
slow
> > >>>> to build and rebuild.
> > >>>>
> > >>>> On Sat, Feb 13, 2010 at 12:03 AM, Mathias Herberts
> > >>>>    wrote:
> > >>>>
> > >>>>
> > >>>>
> > >>>>
> > >>>>> -1
> > >>>>>
> > >>>>> I think Maven is too complex and will lower the adoption of
HBase by
> > >>>>> people today willing to build it.
> > >>>>>
> > >>>>> I would suggest using Ivy for dependency management as was
done in
> > >>>>> Thrift.
> > >>>>>
> > >>>>> Mathias.
> > >>>>>
> > >>>>>
> > >>>>>
> > >>>>>
> > >>>>>
> > >>>>
> > >>>>
> > >>>
> > >>>
> > >>
> > >>
> > >>
> > >
> > >
> >
> >
> > --
> > Connect to me at http://www.facebook.com/dhruba
> >



      


Mime
View raw message