www-mirrors mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Tornoci Laszlo <torl...@xenia.sote.hu>
Subject Re: steps towards better mirroring
Date Sun, 14 Jan 2001 16:35:26 GMT
On Sat, 13 Jan 2001, Brian Behlendorf wrote:

> Hi folks.  It may not seem like it, but the apache.org developers are
> fairly aware that we've busted mirroring in a bad way with our split into
> vhosts.  Fixing that completely (moving all content to be under a single
> mirrorable www.apache.org) is still being discussed, with some good
> reasons not to do it (namely, that there may be publishing processes that
> are incompatible with that, or that there may be content/builds/etc that
> asking the mirror sites to take would be too much).

It is ok to have the projects separately, as long as we get _clear_
and _up-to-date_ instructions on how to mirror them.

> While that's being discussed, what's not as controversial is the idea of
> focusing for the time being http://www.apache.org/dist/, and especially to
> make it non-project-specific and non-httpd-specific.

Great.

> We'd thus be rearranging it to look something like
>
> /dist/codebase/[binaries|*]
...
> One alternative that's been proposed is /dist/project/codebase/,
> i.e. /dist/xml/xerces/ instead of just /dist/xerces/.
...
> Care would be taken to make sure that only the most recent versions of any
> particular branch of the code line would be there, as well as only the
> most recent binary builds.  E.g, today, under /dist/httpd-2.0/, we'd have
> apache-2.0a9, and *perhaps* 2.0a8, but older ones would be shuffled off to
> httpd.apache.org/dist/ and perhaps further under a /dist/old or something.

> This is to try and keep low the total amount of disk space needed to
> mirror the apache.org distributables.

As already mentioned by someone else, this approach saves on disk space,
but forces a new download from all mirrors every time a file is moved.

A good alternative could be something like:

/dist/codebase/version-x/
/dist/codebase/current/source
/dist/codebase/current/bin-architecture-xxx
/dist/codebase/old/

where real files are in version-x/ (src and binaries)
current/whatever is a symlink to the latest src and binaries, while
old/ has only symlinks pointing to older releases

Those who want to provide a full mirror could do rsync the usual way.
The setup is a bit tricky for those who want only the latest files, but
if I understand the rsync man page correctly, syncing the current/whatever
symlinks with the --copy-links option would actually get the latest files
and nothing else. I never tried this myself, but it is an idea worth
considering. This way there is no need to move files on the central
server, so we don't have to re-download anything.

Of course this whole setup breaks, if the symlinks are not updated
properly on the central server. If the apache team cannot guarantee this,
I still vote for a simple /dist/codebase/version-x/ scheme, without any
/old dir and file relocations. With proper exclusion criteria one can
still do a partial mirror.


                              Yours: Laszlo
-- 

----------------------------------------------------------------------
Laszlo L Tornoci                            Inst Pathophysiology
E-mail: torlasz@xenia.sote.hu               Semmelweis Univ Med School
        torlasz@net.sote.hu                 Nagyvarad ter 4.
fax:    (36-1)-210-4409                     Budapest, H-1089, Hungary
----------------------------------------------------------------------


Mime
View raw message