www-infrastructure-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Santiago Gala <santiago.g...@gmail.com>
Subject Re: [scm] Use case: Continuous integration
Date Sat, 01 Mar 2008 13:52:37 GMT

El sáb, 01-03-2008 a las 14:06 +0200, Jukka Zitting escribió:
> Hi,
> 
> Continuous integration tools would probably be worth a whole topic of
> their own, but since they are related to version control I'm taking
> them up within this scope as well.
> 
> Use case: Someone (either within or outside Apache) sets up a
> continuous integration system and wants to get the latest sources from
> the source repository. Optimally the system would automatically
> compile, package, and test the sources after each commit, but hourly,
> daily, or weekly builds would also be acceptable depending on the
> scope of the tests and available computing resources.
> 
> Variants and implementation options:
> 
> 1) Push-based CI: The SCM system would notify the CI system of all
> source changes so that the system can start processing the changes as
> soon as possible. Currently the best way to achieve this would
> probably be to subscribe the CI system to the relevant -commits
> mailing list, but other alternatives might also be possible. Assuming

There is currently a trend to have feeds associated with SCM servers.
This is fully included in gitweb and "hg serve", and requires a script
(bzr-feed http://bzr.mfd-consult.dk/bzr-feed-global.atom ) for bzr. In
subversion I have used the free service at
http://subtlety.errtheblog.com/ to get something like the ASF public
repo feed: http://subtlety.errtheblog.com/O_o/2d.xml This is useful not
only for Continuous Integration, but also for having a better monitoring
of changes, see how Sam Ruby has integrated a planet of venus
repositories in planet intertwingly, for instance. It would be great
that our subversion could do that "cheaply", via
http://subversion.tigris.org/servlets/ReadMsg?listName=dev&msgNo=117974
or contrib/hook-scripts/svn2rss.py and integrate this feed serving with
feed self-discovery or in a separate URL. Publicizing it could save a
fair amount of load to the repository.

> there are enough computing resources either at Apache or an external
> CI lab for such potentially high-frequency integrations, would the
> associated load on our Subversion server be acceptable? If not, how
> could we resolve such issues?

I'm not sure about the issue. you mean that the CI server would do
checkouts and delete the data for each run? if this is the case, and
this is the only case where I can see a potential high load, I'd say the
alternative of having some sort of incrementally updated clean
sort-of-distributed repository from where to pull seems reasonable.

> 
> 2) Pull-based CI: The CI system regularly polls the source repository
> for new changes. Again, assuming enough CI computing resources, what
> would be the smallest acceptable polling interval from the perspective
> of our Subversion server. Are there other considerations that such CI
> systems should be aware of when accessing the source repository?
> 

feeds are "polling", actually, or push via polling. We could publish our
own, static and with creation triggered from commit emails, so that
people can subscribe to it instead of polling themselves.

> Related to this and some of the other raised issues, would it be a
> good idea to consider one or more read-only mirrors of our Subversion
> repository. I'm not sure how feasible such mirroring would be with
> current Subversion, but in the long run something like that seems more
> scalable and fault-tolerant than upgrading a single svn server.
> 
> BR,
> 
> Jukka Zitting
-- 
Santiago Gala
http://memojo.com/~sgala/blog/


Mime
View raw message