ant-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Antti Luoma" <anlu...@gmail.com>
Subject Re: Syncing very large directories
Date Fri, 25 Jan 2008 06:56:56 GMT
Hi,

I did take a look from sources and noticed the basic java File API, I think
there might be some optimization (like getting rid of unnecessary vector
creation). Also caching src folder contents (is maybe storing contenst to
outputstream directly and reading from there) might improve it? This of
course is if you sync SAME directories..  Or maybe somebody has an even
better idea?

I'm stuck with windows platform (due to sdk that I'm using).. But i'll think
I give rsync a try (even though used linux for years this is something that
I haven't worked with before). It might even work better on my case (client
/ server model..). Only side effect it that i'll need another prorgram and
more setup to make this system rock (not so easy to transfer to another
machine, that would have been the best part with using build in ant tasks).

But thanks!


-- 
-Antti-

2008/1/24, Dominique Devienne <ddevienne@gmail.com>:
>
> On Jan 24, 2008 3:53 AM, Antti Luoma <anluoma@gmail.com> wrote:
> > Hi,
> >
> > I am using ant 1.7.0 <sync> task to sync 2 very large directories. It
> > currently takes too much time to do this (to be usable).
>
> Sorry Antti, but there's no free lunch. Either you want to sync, in
> which case you need to scan both the source and the destination
> folders, or you don't. The whole point of sync is that things can
> change in either folders, so you must scan them to know what.
>
> <sync> uses a simple minded algo, using existing Java File APIs, which
> aren't that fast, especially with remote file systems. I don't see how
> to speed it up without compromising the goal of syncing things.
>
> rsync OTOH is a much smarter approach, but relies on a client-server
> approach where both the source and destination must be rsync-enabled
> to be really fast (AFAIK), and not very win32-friendly (I could not
> make it work with UNC paths for examples).
>
> 15 hours is a long time. That must be one mighty big folder... --DD
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: user-unsubscribe@ant.apache.org
> For additional commands, e-mail: user-help@ant.apache.org
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message