lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Clemens Marschner" <>
Subject Re: [LARM] using SEDA, pros and cons
Date Tue, 18 Feb 2003 13:12:46 GMT
Couldn't look at your code yet, but:

AFAIK Seda was developed for asynchronous I/O, which would mean a redesign
of the central FetcherTask class. If every thread downloads 50 files at
once, you only need a couple of them in parallel to saturate the network


----- Original Message -----
From: "David Worms" <>
To: <>
Cc: "Avalon users" <>
Sent: Monday, February 17, 2003 11:32 PM
Subject: [LARM] using SEDA, pros and cons

> In order to move our project to the next level, it has been discussed
> the use of an SEDA architecture.
> With the help of the Excalibur / Event library, I created a simplified
> version of Larm which use SEDA as its backbone architecture. The goal
> was not to make it pretty or even working, but to see how each
> application components could feet in an event stage architecture.
> The great thing I notice is how flexible the application become. it is
> extremely easy to map each stage with the others. One of the features
> of LARM is the ability to have different sources( db, web, filesystem,
> ... ), process them, and store them (lucene index, log, ...). This
> seems easily achieveable with SEDA. Also clustering the LARM could be
> done through a specific stage implementation.
> This was for the pros. However, it will be great to get some feedback
> because I am really not sure on how to deal with SEDA. here is some
> problems I am facing.
> - In order for the crawler to be efficient, I had to raise the number
> of threads, but from what I read in the past, only one or two threads
> should be used in a SEDA environment.
> - Also, it looks to consume a lot of memory, which could be due to the
> number of messages put into the queue.
> Clemens, and others, please have a look of it, and give me some
> feedback.
> David.
> ---------------------------------------------------------------------
> To unsubscribe, e-mail:
> For additional commands, e-mail:

To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message