maven-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Curtis Rueden <ctrue...@wisc.edu>
Subject Re: System-wide, read-only repository
Date Wed, 30 Oct 2013 15:51:12 GMT
Hi Roy,

> zfs has built in de-duplication.

ZFS sounds awesome in theory but have you actually tried it? If so, how is
it working for you? In particular, how is the performance?

-Curtis


On Wed, Oct 30, 2013 at 10:43 AM, Lyons, Roy <Roy.Lyons@cmegroup.com> wrote:

> :) but like I said, you wouldnt worry about the space if it was all on
> zfs.  zfs has built in de-duplication.  you could have 2000 local maven
> repos and probably not fill your disk since most of it has to do with
> duplicate jars and such.
>
> On 10/30/13 10:37 AM, "Curtis Rueden" <ctrueden@wisc.edu> wrote:
>
> >Hi all,
> >
> >There is plenty of room for improvement regarding reuse of Maven's local
> >repository cache. Releases in particular are supposed to be immutable so
> >once they are downloaded they could go into a read-only tier as suggested
> >by Stephen. Inventing such a scheme to reuse large portions of the repo
> >cache would be of great benefit to the Maven community.
> >
> >E.g.: the recommended CIS strategy is for every job to use its own local
> >repo cache, which becomes very large. My Jenkins has dozens of Maven build
> >jobs and I cannot afford the bloat; my Jenkins backups are huge enough
> >already. So what I do instead is limit my Maven Jenkins node to a single
> >executor, which is a real waste on a 16 core machine. Much better would be
> >if the jobs could share the bulk of the repo cache.
> >
> >So it's definitely an itch, but not quite itchy enough for anyone to
> >scratch yet...
> >
> >Regards,
> >Curtis
> > On Oct 30, 2013 8:35 AM, "Mark H. Wood" <mwood@iupui.edu> wrote:
> >
> >> On Wed, Oct 30, 2013 at 10:18:49AM +0100, Matthieu Moy wrote:
> >> > Barrie Treloar <baerrach@gmail.com> writes:
> >> >
> >> > > On 29 October 2013 23:56, Lyons, Roy <Roy.Lyons@cmegroup.com>
> wrote:
> >> > >> Unfortunately, you will always have something in
> >>$HOME/.m2/repository
> >> > >> because that's how maven works.
> >> > >>
> >> > >> Can I suggest perhaps that you use zfs for deduplication in /home?
> >> > >> Otherwise, you can add something like
> >> > >
> >> > > Or give them more disk space - isn't this stuff meant to be cheap
> >> now-a-days?
> >> >
> >> > Local disk space is cheap. NFS-shared, RAID & backed-up disk space,
> >>less
> >> > so. I can live with a few Gb of waste, but I was just wondering
> >>whether
> >> > we could do any better.
> >>
> >> Disks are cheap.  But not free.  Running the procurement gantlet is
> >> not free.  Downtime to install new storage is not free.  Lord knows
> >> that additional backup tapes are not free, not even cheap.  Longer
> >> backup windows are not free.  Throwing storage at the problem is often
> >> a reasonable choice, but it's also reasonable to always ask if there
> >> isn't a better way.
> >>
> >> Sorry, I've been aching to write that for a long time....
> >>
> >> --
> >> Mark H. Wood, Lead System Programmer   mwood@IUPUI.Edu
> >> Machines should not be friendly.  Machines should be obedient.
> >>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@maven.apache.org
> For additional commands, e-mail: users-help@maven.apache.org
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message