hadoop-mapreduce-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Robert Evans <ev...@yahoo-inc.com>
Subject Re: Picking up local common changes in mr
Date Fri, 19 Aug 2011 16:51:43 GMT
One thing to be aware of is that with -SNAPSHOT at the end of the version Maven will start
looking at dates.  So if you have a 0.23.0-SNAPSHOT that you personally modified/built in
your .m2 repository and go to build something that depends on it.  If the nightly build has
pushed it to the apache repo after you built your version maven might download the "newer"
version replacing your changes.  If you changes impact multiple components then your choices
are to always build the entire project (or at least the subset that has dependent changes)
or always build with -o after your initial build/install.

--Bobby

On 8/19/11 11:41 AM, "Matt Foley" <mfoley@hortonworks.com> wrote:

Thanks for the nice clear statement, Alejandro.
--Matt

On Thu, Aug 18, 2011 at 4:40 PM, Alejandro Abdelnur <tucu@cloudera.com>wrote:

> This is handled by maven reactor.
>
> When your run Maven in a multimodule project (like we have), all modules
> that are part of the build (from the dir where you are) down are used for
> the build/test/packaging, all modules that are not part of the build are
> picked up from .m2/repo.
>
> For example
>
> "cd trunk/hadoop-mapreduce;mvn compile" uses hadoop-common & hadoop-hdfs
> from m2/repo
>
> "cd trunk;mvn compile" uses hadoop-common, hadoop-hdfs, hadoop-mapreduce
> from the build.
>
> HTH
>
> Thxs.
>
> Alejandro
>
>
> On Thu, Aug 18, 2011 at 4:35 PM, Matt Foley <mfoley@hortonworks.com>
> wrote:
>
> > Since we put all the effort into "un-splitting" the components, shouldn't
> > we
> > have a switch
> > that causes, eg, the MAPREDUCE build to pick up artifacts from COMMON and
> > HDFS builds
> > in specified sibling directories, without using m2 as an intermediary?
> >
> > Of course it should respect dependencies (via maven) so that if HDFS
> source
> > has been modified,
> > the HDFS artifacts will also be rebuilt before MAPREDUCE uses them :-)
> >
> > --Matt
> >
> > On Thu, Aug 18, 2011 at 3:30 PM, Giridharan Kesavan <
> > gkesavan@hortonworks.com> wrote:
> >
> > > Hello,
> > >
> > > Its the same -Dresolvers=internal for the ant build system; For the
> > > maven/yarn build system as long as you have the latest common jar in
> > > the m2 cache its going to resolve common from the maven cache. If not
> > > from the apache maven repo. You can force the builds to use the cache
> > > by adding -o option. (offline builds)
> > >
> > > Thanks,
> > > Giri
> > >
> > > On Thu, Aug 18, 2011 at 3:19 PM, Eli Collins <eli@cloudera.com> wrote:
> > > > Hey gang,
> > > >
> > > > What's the new equivalent of resolvers=true in the new MR build? ie
> > > > how do you get a  a local common change to get picked up by mr?
> > > >
> > > > Thanks,
> > > > Eli
> > > >
> > >
> >
>


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message