lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From stephane vaucher <vauc...@LUB.UMontreal.CA>
Subject Re: LARM work
Date Sat, 26 Oct 2002 23:44:38 GMT
Hi Clemens,

I'm new to both Lucene & LARM and am wondering if you've been inspired 
by some code from JoBo(http://sourceforge.net/projects/jobo/) to 
implement some of LARM functionality. Currently, I'm using (a highly 
modified version of) JoBo to fetch pagesfrom a few sites. I don't think 
it's fantastic, but it gets the job done.

I know for one, that it has the feature of adding a timeout and limiting 
bandwidth.

Stephane


Clemens Marschner wrote:

>Hi,
>
>now I'm finally done and already thinking about the new LARM developments.
>
>I see  the following priorities:
>1) Make all properties for a crawl configurable
>2) Update documentation and make LARM more public (and probably get more
>developerts on the project)
>3) Implement the "politeness" stuff. Your idea with the
>one-thread-per-server seems good to me
>4) implement "sources" and "drains" and rework the processing pipelines to
>be able to split up document processing pipelines into several steps
>5) make the "FetcherTask.run()" more generic by putting most of it into the
>processing pipelines
>
>Regarding 1) I will try to do a spike with Avalon Phoenix, from which I
>still believe that it perfectly matches what I've outlined in the
>"Configuration RFC" mail. It provides XML configuration, a split up between
>architecture (assembly.xml) and properties (config.xml), management of
>startable/stoppable components, management of dependencies between
>components, a management console, etc. The only glitch: the docs are still
>painful, there are (afaik) no other resources on the net other than the
>Avalon website, and it takes some time to understand the concepts. That
>again lets entry costs for other people raise.
>
>Clemens
>
>
>
>----- Original Message -----
>From: "Otis Gospodnetic" <otis_gospodnetic@yahoo.com>
>To: "Clemens Marschner" <Clemens.Marschner@internet.lmu.de>; "Mehran Mehr"
><mehran@sharif.edu>
>Sent: Tuesday, October 15, 2002 7:43 PM
>Subject: LARM work
>
>
>>Mehran, Clemens,
>>
>>Just curious, now that you are done with studies, Clemens, what is next
>>for LARM?
>>
>>Otis
>>
>>
>>__________________________________________________
>>Do you Yahoo!?
>>Faith Hill - Exclusive Performances, Videos & More
>>http://faith.yahoo.com
>>
>
>
>
>--
>To unsubscribe, e-mail:   <mailto:lucene-dev-unsubscribe@jakarta.apache.org>
>For additional commands, e-mail: <mailto:lucene-dev-help@jakarta.apache.org>
>
>



--
To unsubscribe, e-mail:   <mailto:lucene-dev-unsubscribe@jakarta.apache.org>
For additional commands, e-mail: <mailto:lucene-dev-help@jakarta.apache.org>


Mime
View raw message