lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Clemens Marschner" <c...@lanlab.de>
Subject Re: Development plans for Lucene?
Date Wed, 30 Oct 2002 19:33:32 GMT
Otis, Kelvin and me have been discussing how we could leverage Lucene on a
next level. We have some components in the sandbox (LARM Webcrawler, Indyo
indexing framework) that have to be weaved together with Lucene.

This could end up in a real search engine server. I call it "Lucene Advanced
Retrieval Machine" for myself right now, 'cause then I can stick with the
LARM acronym... :-)

This will be a server that gets data from somewhere (i.e. the Web or the
file system), does processing steps on it (i.e. extracting text from HTML or
PDF) and puts the results into a Lucene index. The whole process forms an
assembly line, with data sources at the beginning and a storage at the end.
A source is an active component that decides when to put a URL or a document
into the queue (i.e. because there's a notification if the doc has changed,
if the stored doc is obsolete, or if a recrawl is said to be necessary).

This assembly line can also be broken up (in serial or in parallel) such
that different steps of it can be executed on more than one machine. As
such, the storage could be a JMS queue (or even a topic) that is filled in
batch mode. On the other side, the JMS then is the data source.

There are lots of construction sites, though. The architecture must be solid
and efficient. The crawler has to be made more scalable. We're also
discussing on how we can save time implementing server related stuff and
configuration - taking the Avalon framework is (probably a good) option, but
no one of us has used it before.

Clemens


----- Original Message -----
From: "Peter Mularien" <pmularien@deploy.com>
To: "Lucene Developers List" <lucene-dev@jakarta.apache.org>
Sent: Wednesday, October 30, 2002 6:10 PM
Subject: Development plans for Lucene?


> Hi -
>
> I was just wondering what is the current development plan status for
> Lucene? I have been monitoring the developer's list for some time, and
> have seen very little in the way of CVS commits. I know that may not be
> an accurate barometer of development activity, but there it is.
>
> Is there any plan for continuing active development of Lucene? How many
> people are working on it? Why aren't there more, and if so, how can we
> recruit them?
>
> I am willing to help contribute, but without some other active
> developers, I'm not sure how useful it will be.
>
> The frustrating thing is that from the users list, we all know that
> there is a significant amount of interest in the product, and some
> feature requests, etc.
>
> Thanks
> Peter Mularien
> Deploy Solutions
>
>
>
> --
> To unsubscribe, e-mail:
<mailto:lucene-dev-unsubscribe@jakarta.apache.org>
> For additional commands, e-mail:
<mailto:lucene-dev-help@jakarta.apache.org>
>


--
To unsubscribe, e-mail:   <mailto:lucene-dev-unsubscribe@jakarta.apache.org>
For additional commands, e-mail: <mailto:lucene-dev-help@jakarta.apache.org>


Mime
View raw message