lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Clemens Marschner" <c...@lanlab.de>
Subject Re: [LARM] using SEDA, pros and cons
Date Tue, 18 Feb 2003 13:12:46 GMT
Couldn't look at your code yet, but:

AFAIK Seda was developed for asynchronous I/O, which would mean a redesign
of the central FetcherTask class. If every thread downloads 50 files at
once, you only need a couple of them in parallel to saturate the network
interfaces.

Clemens


----- Original Message -----
From: "David Worms" <david@simpledesign.com>
To: <lucene-dev@jakarta.apache.org>
Cc: "Avalon users" <avalon-users@jakarta.apache.org>
Sent: Monday, February 17, 2003 11:32 PM
Subject: [LARM] using SEDA, pros and cons


>
> In order to move our project to the next level, it has been discussed
> the use of an SEDA architecture.
>
> With the help of the Excalibur / Event library, I created a simplified
> version of Larm which use SEDA as its backbone architecture. The goal
> was not to make it pretty or even working, but to see how each
> application components could feet in an event stage architecture.
>
> The great thing I notice is how flexible the application become. it is
> extremely easy to map each stage with the others. One of the features
> of LARM is the ability to have different sources( db, web, filesystem,
> ... ), process them, and store them (lucene index, log, ...). This
> seems easily achieveable with SEDA. Also clustering the LARM could be
> done through a specific stage implementation.
>
> This was for the pros. However, it will be great to get some feedback
> because I am really not sure on how to deal with SEDA. here is some
> problems I am facing.
>
> - In order for the crawler to be efficient, I had to raise the number
> of threads, but from what I read in the past, only one or two threads
> should be used in a SEDA environment.
>
> - Also, it looks to consume a lot of memory, which could be due to the
> number of messages put into the queue.
>
> Clemens, and others, please have a look of it, and give me some
> feedback.
>
> http://67.116.155.180/~wdavidw/stage.zip
>
> David.
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: lucene-dev-unsubscribe@jakarta.apache.org
> For additional commands, e-mail: lucene-dev-help@jakarta.apache.org
>


---------------------------------------------------------------------
To unsubscribe, e-mail: lucene-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: lucene-dev-help@jakarta.apache.org


Mime
View raw message