ant-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Xavier Hanin <xavier.ha...@gmail.com>
Subject Re: Ivy Indexer
Date Sat, 14 Nov 2009 15:42:05 GMT
One thing I'm not sure to fully understand: it seems that you plan to store
the index on the client (say the developer's box), according to your example
with dir="${ivy.settings.dir}/index". But it also seems like every client
will have the responsibility to maintain the index, is that right? Sound's
strange, there's probably something I miss?

Having already thought about that previously, I'm not sure having using an
index on the client is the best way to go. It requires either to maintain
the index on the client, which can take a very long time, or to download the
index on the client (maven indexes approach IIUC), which is not optimizing
the network transport, such indexes can become very large if you want a lot
of useful information. So IMHO this should be something part of a repository
manager, providing a remote API (I thought of a rest API) queried by the
clients (ivy, ...). The problem with that approach is that you require to
install a new server side tool on your repository, which is not as easy as
using a basic shared filesystem. But I think nowadays most people already
use dedicated resource to host their own module repository (http server,
svn, ...), so having to deploy a war somewhere to have quick search and
advanced features like the organize dependencies you suggest is a reasonable
tradeoff IMO.

What do you think?

Xavier

On Sat, Nov 14, 2009 at 16:08, Jon Schneider <jkschneider@gmail.com> wrote:

> That is exactly right.  It is also the first step in two of my other future
> goals:
>
> 1.  Unused dependency analysis.  I realize this won't ever be perfect since
> there is no limit to the way dependencies can be used (sometimes
> exclusively) in specific metadata like Spring context files, but
> nevertheless, it would be a good watermark for unused dependencies.
>
> 2.  "Organize Dependencies", hooking the Eclipse "search engine" API, if I
> can find a public facing API interface to it.  It would be awesome to proxy
> the Organize Imports functionality in Eclipse with this add dependency (and
> remove unnecessary dependency) functionality.
>
> Jon
>
> 2009/11/13 Nicolas Lalevée <nicolas.lalevee@hibnet.org>
>
> > On Friday 13 November 2009 10:24:56 Gilles Scokart wrote:
> > > Seems nice.  But I'm not sure I understand what it will be used for.
> > > What would be the user interface to read the index ?
> >
> > The use case is pretty simple: I work on a project with no dependency.
> Then
> > I
> > know that there some cool stuff in commons-io, I want to use FileUtils
> for
> > instance. More than trying to find the exact organisation+module name
> > (commons-io/commons-io or apache/commons-io or
> > org.apache.commons/commons-io
> > or org.apache.commons/io, etc....), I would open a search windows where I
> > put "FilesUtils" in a search field, and it would find the proper
> > organisation
> > and module names. Then there would be a "add dependency" button which
> will
> > add it to the ivy.xml of the project.
> >
> > Nicolas
> >
> >
> > >
> > >
> > > Gilles Scokart
> > >
> > >
> > > 2009/11/11 Jon Schneider <jkschneider@gmail.com>
> > >
> > > > I've been thinking about IVYDE-134 (Quick Search feature for
> > dependencies
> > > > in
> > > > repositories) and related IVY-866. If we add support for the Nexus
> > > > Indexer (which would be nice in its own right), we would still be
> > lacking
> > > > this feature for Ivy repositories. Also, what about ivysettings whose
> > > > default resolver is a chain resolver of a Maven repository and an Ivy
> > > > repository? In
> > > > this case, without some all-encompassing index, the quick search
> > feature
> > > > would find Java types in only the Maven repository within the chain
> > > > resolver, which I think would be counterintuitive to a user.
> > > >
> > > > My first thought was to build an extension to Nexus or Archiva for
> Ivy,
> > > > but somehow I just really dislike the idea of making an otherwise
> > > > stateless repository stateful (or should I say, having a manager,
> > however
> > > > thin, continuously running to proxy modifications to the repository).
> > > > Also, these two products are so Maven-centric (due to their intended
> > use)
> > > > that any extension would amount to an abuse of their intended use.
> > > >
> > > > So my compromising proposal is centered around a Lucene index that
> > should
> > > > be
> > > > modified (1) whenever a deliver/publish/install task is ran. Also,
> > since
> > > > nothing stops a repository administrator from manually
> > > > deleting/adding/updating files in the repository, we should provide
> (2)
> > a
> > > > new <ivy:index> task.
> > > >
> > > > (1) is accomplished through a new resolver type extending from
> > > > ChainResolver
> > > > that proxies publishing to its delegate resolvers, indexing the
> > published
> > > > artifacts in the process. As an example, adding this proxy would look
> > > > like this in ivysettings.xml:
> > > >
> > > > <resolvers>
> > > > <indexed name="indexable" index="${ivy.settings.dir}/index">
> > > > <filesystem name="1">
> > > > <ivy
> > > >
> > pattern="${ivy.settings.dir}/[organisation]/[module]/ivy-[revision].xml"/
> > > >> <artifact
> > > >
> > > >
> > pattern="${ivy.settings.dir}/[organisation]/[module]/[type]/[artifact]-[r
> > > >evision].[ext]"/> </filesystem>
> > > > <!-- other resolvers here... -->
> > > > </indexed>
> > > > </resolvers>
> > > >
> > > > (2) allows a repository administrator to force clean the index via an
> > Ant
> > > > task when it is known to be stale. It also provides an alternative to
> > > > using the proxy mechanism described in (1); the index task could be
> run
> > > > periodically (e.g. nightly) as a task on a continuous integration
> tool.
> > > >
> > > > The index task itself explores the repository, opening jars and
> listing
> > > > the fully qualified types found in each jar in the index and
> > associating
> > > > these types with a particular ModuleRevisionId. With the code I have
> > > > written so far, I have been able to index up to 10,000 jars in less
> > than
> > > > 10 seconds when the index task is running against a repository on the
> > > > same machine (indexing a repository through a network path slows down
> > > > considerably).
> > > >
> > > > IvyDE can then search for types against the optimized Lucene index,
> > > > making it very fast.
> > > >
> > > > Thoughts on this approach?
> > > > Jon
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: dev-unsubscribe@ant.apache.org
> > For additional commands, e-mail: dev-help@ant.apache.org
> >
> >
>



-- 
Xavier Hanin - 4SH France - http://www.4sh.fr/
BordeauxJUG creator & leader - http://www.bordeauxjug.org/
Apache Ivy Creator - http://ant.apache.org/ivy/

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message