Mailing-List: contact dev-help@ant.apache.org; run by ezmlm
Precedence: bulk
Reply-To: "Ant Developers List" <dev@ant.apache.org>
Received-SPF: softfail (athena.apache.org: transitioning domain of
 nicolas.lalevee@hibnet.org does not designate 216.86.168.183 as permitted
 sender)
Content-Type: text/plain; charset=iso-8859-1
Mime-Version: 1.0 (Apple Message framework v1084)
Subject: Re: An new kind of cache for Ivy
From: =?iso-8859-1?Q?Nicolas_Lalev=E9e?= <nicolas.lalevee@hibnet.org>
In-Reply-To: <42188812-DAF8-4923-960D-D96147AFC3BD@hibnet.org>
Date: Fri, 29 Apr 2011 00:07:00 +0200
Content-Transfer-Encoding: quoted-printable
Message-Id: <CA73966C-E565-4B35-BA45-6475284B6BDC@hibnet.org>
References: <42188812-DAF8-4923-960D-D96147AFC3BD@hibnet.org>
To: "Ant Developers List" <dev@ant.apache.org>


Le 14 avr. 2011 =E0 22:38, Nicolas Lalev=E9e a =E9crit :

> Hi,
>=20
> I am starting to have Ivy be able to understand some Eclipse =
updatesite. I am now facing issues with the size of the downloaded =
metadata. So the next step is to put these files in a cache.
>=20
> I had the same issue with the obr resolver. I did resolve this but =
with a kind of work around. I used the repositoryCacheManager associated =
with the resolver and make the obr xml descriptor an artifact in a =
special organisation+module. See the method init in OBRResolver [1].
>=20
> For this kind of metadata, which are a sort of per "repository" or per =
resolver data rather the module data, we may need a specific cache for =
repository level data. I think this would also be helpful if one day we =
want to support some maven repository index [2].
>=20
> But then I am wondering how should I manage the key for the entry in =
the cache. I am starting to think of a hash of the url of the resource =
to store. Would it be sufficient ? Should I also add the resolver name, =
or the resolver type ?

Finally I implemented a new method in the RepositoryCacheManager: =
downloadRepositoryResource. It will put in cache any resource associated =
with a repository. Since these resources are associated with a =
repository, I though that the proper place of their cache is next to the =
cache of the module's artifact. So no need for another kind of cache.

About the implementation, I reused some code of the =
DefaultRepositoryCacheManager, so that the store of that kind of =
resource is stored like any other module artifact. The "organisation" of =
such resource will be then "_repository_metadata_", and the module name =
will be the SHA-1 of the resource name (the url for an URLResource) =
encoded in hexadecimal (just like does git about commits id actually).

I made the caching quite aggressive. If a resource is not found, we are =
caching that we didn't found it. And we'll check for updates of these =
resources (or check that a missing resource still miss) not more than =
every hour. It's obviously configurable, see CacheResourceOptions. Maybe =
we should also have a "magic" property like =
"ivy.repository.resource.cache.ttl".

It is still missing some unit test dedicated to that new method. The =
tests with the OBRResolver, the MirroredURLResolver and the =
UpdateSiteLoader, which are now using that new caching method, shows =
that is works quite well.

Nicolas


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@ant.apache.org
For additional commands, e-mail: dev-help@ant.apache.org