ant-ivy-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Tim Brown <tpbr...@gmail.com>
Subject Re: Building a module only when changed
Date Wed, 20 Jan 2010 16:10:00 GMT
One thing we did to speed the process of building everything with each
change was to put Ivy cache, source checkouts, and build tooling all
on a large (12G) ram disk.

Absolutely huge difference.  We're running a good 1,000+ builds a day
(of this type alone).

There are other approaches and patterns to apply if performance is
still an issue, but that's a topic for a conference presentation. If I
ever get off my bum and submit ;-)

~Tim

On Jan 20, 2010, at 1:46 PM, Shawn Castrianni <Shawn.Castrianni@halliburton.com
 > wrote:

> I agree.  We have 80 modules in our dependency hierarchy and used
> QuickBuild2 to do continuous on each of the modules.  But as Tim
> pointed out, we got into compatibility issues when IVY detected
> conflicts and picked one as the winner.  We have since gone to doing
> 4 full 80 module builds in dependency order 4 times per 24 hour
> period.  We still use QuickBuild2 because I believe it is the most
> flexible and powerful out of the box and their plugin API, due out
> later this month, will only make it better.
>
> ---
> Shawn Castrianni
>
> -----Original Message-----
> From: Tim Brown [mailto:tpbrown@gmail.com]
> Sent: Wednesday, January 20, 2010 8:42 AM
> To: ivy-user@ant.apache.org
> Cc: Timothy.Aston@ca.ibm.com
> Subject: Re: Building a module only when changed
>
> +1 to that.
>
> We use Cruise to trigger builds based upon SCM changes (amongst other
> things).
>
> However, I _have_ setup builds where it determined if a particular
> module
> has been built since it's last change.  This was using Subversion,
> so excuse
> the svn commands ;-)
>
>
>   - svn info the module directory, extract out the last changed
> revision
>   - ivy findrevision that module, that revision into your repo
>   - if it doesn't exist, build it.
>
> That said, I don't recommend going with this approach.  In my
> experience,
> the gains of not just rebuilding everything outweigh the expected
> speed
> improvement.  You start finding cases where X hasn't changed, but
> depends on
> Y.  Y changes, and is no longer compatible with X.
>
> You get into the impact-zone approach of testing, which adds to the
> overhead
> of determining if you need to build, etc.
>
> My $0.02.
>
> ~Tim
>
>
> On Wed, Jan 20, 2010 at 6:33 AM, Gareth Western <gareth@garethwestern.com
> >wrote:
>
>> This sounds more like an issue for your build system rather than the
>> software. I think what you're looking for is a "Continuous
>> Integration" server such as Hudson, CruiseControl, or Bamboo. There
>> are loads of others, but those are the 3 I hear about most
>> frequently.
>>
>> These typically monitor your SCM for changes and then invoke a build
>> mechanism (Ant files, Maven, custom schell scripts, etc.), and then
>> publish the resulting artifacts somewhere (upload to FTP, publish in
>> Ivy, etc).
>>
>> Hudson: http://hudson-ci.org/
>> CruiseControl: http://cruisecontrol.sourceforge.net/
>> Bamboo: http://www.atlassian.com/software/bamboo/
>>
>> Is that what you meant?
>>
>> On Wed, Jan 20, 2010 at 2:17 PM, Timothy Aston <Timothy.Aston@ca.ibm.com
>> >
>> wrote:
>>>
>>>
>>> I'm wondering if anyone can advise on techniques for having a module
>> build
>>> only if there are changes.  i.e. if the modified dates on any
>>> files in a
>>> particular fileset (the source files) are later than the
>>> publication date
>>> of the latest version of the module.  I have a feeling this may
>>> require
>>> ant-contrib, which is OK as our build is already using that for
>>> other
>>> reasons.
>>>
>>>
>>> -Tim
>>
>
> ----------------------------------------------------------------------
> This e-mail, including any attached files, may contain confidential
> and privileged information for the sole use of the intended
> recipient.  Any review, use, distribution, or disclosure by others
> is strictly prohibited.  If you are not the intended recipient (or
> authorized to receive information for the intended recipient),
> please contact the sender by reply e-mail and delete all copies of
> this message.

Mime
View raw message