directory-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Berin Loritsch <blorit...@apache.org>
Subject Re: [seda] Re: SEDA
Date Wed, 14 Jan 2004 16:02:44 GMT
Alex Karasulu wrote:
> Hello again,
> 
> 
>>From: rwallace@thewallacepack.net
>>Quoting Tim Walker <twalker@merc-int.com>:
>>
>>
>>>This is very interesting. Is the 'Avalonized version of Seda' a rewrite? Any
>>>thoughts on the motivation for
>>>the rewrite in Eve?
> 
> 
> Tim I have looked at the event stuff in Excalibur in the past and 
> should probably revisit it before responding but well I'm runing on
> low fuel now adays so I'll just state that it was more complex for
> my needs.  Also I think it was geared to use sandstorm I think.
> 
> My view of SEDA may be a bit oversimplified.  I view a stage as 
> a event source or sink.  Events are enqueued onto the event queues
> of stages.  Enqueue predicates associated with the stage are used
> to determine if the enqueue operation should occur or not.  Stages
> have a thread pool.  A stage handler dequeues events delegating 
> processing to a worker thread.  The worker thread processes the 
> event possibly generating other events which are handled by other
> stages.

A stage is a specified location where events are processed and generated.
The event queues are used to connect these stages together.  This is the
core principle of SEDA style implementation.  Each stage is essentially
a component, and they are wired together via a common interface.

> Now these APIs I looked at had all the fancy stuff like setting
> up the source to sink mapping and some other features of a SEDA.
> I did not want to get carried away with this in Eve and just
> implemented simple stages without the bells and whistles.  The
> goal was to simply have staged processing.  The SEDA stuff I have
> encountered in the past left me with the feeling that connection 
> handling with non-blocking IO was too integrated with the simple 
> structures needed to implement SEDA.  I wanted to decouple the
> connection handling from the fact that the architecture is based
> on staged processing of requests.  Really the way you handle 
> connection establishment, reads, writes and droping connections
> will be application and implementation specific so why mix the
> two together.  Also don't presume you have to be non-blocking 
> to implement SEDA.  With continuations you can use blocking sockets
> to multiplex IO handling.  So the less presumptions we make the
> more useful the API.  Also Eve is complex enough so what
> ever simplifications I can make wrt the implementation without
> loosing the benefits the more maintainable the code will be.

The stuff over at Excalibur is actually three distinct sections:

1) The core event API which includes the queues, listeners, and
    predicates/interceptors.

2) The commmand architecture which is a specialized Stage that is
    designed to run arbitrary commands in a set number of threads.

3) Managed pool implementation which uses the command architecture
    to schedule the pool maintenance tasks.

The whole system uses Doug Lea's excellent concurrency tools which
will be made part of JDK 1.5 in some form.  This is great news,
but it also lets you know that the thread pool implementation is
done by someone who knows what they are doing.

What is there will allow you to build a SEDA container on top of
it, but it is only partially there.

> Take for example the code associated with dynamic source->sink
> mappings.  These are all nice features to have when you want to
> dynamically inject a new instrumentation stage between two stages
> to measure the throughput or whatever.  Why bother with it if the
> relationships between your stages are to be static.  Also if 
> one wanted to add an extra stage simply changing your container 
> configuration would allow you to do so with relative ease and
> if designed properly it may be a configuration change not a code
> change.  Lot's of IoC containers like Merlin will allow you to 
> do this with a restart.  Merlin may even get to the point soon
> where it can do it dynamically further lessening the need to have
> source->sink dynamic mappings.

The purpose of the event queues has little to do with dynamic architecture--
although that is an additional benefit of this approach.  The purpose
of the event queues is to manage load.  The best way to look at it is
mass production.  The queues are like the conveyor belts that move the
work from machine to machine (stage to stage).  More work gets done
overall because more things are being processed at the same time.

How many factories reroute the conveyor belts dynamically?  If there
are multiple routes, the machines will make a decision in real time
as to which route the piece of work should go.

> BTW Berin Loritsch from Avalon probably can explain the Excalibur 
> stuff better so I have CC'd him in case he wants to plug in.  Note 
> also that Excalibur may serve your needs - it's good stuff but I
> was looking for only half of it.

I'll do my best.

>>The events package isn't so much a rewrite as it is a different approach using
>>the same ideas.  I'm no expert on the events package in excalibur but from what
>>I understand it is meant as a more general solution following the Avalon design
>>philosophy (i.e. IoC, etc.).

The event package allowed better IoC, and much less class entanglement than the
student project that Matt Welsh put together.  He is an incredibly smart person,
but typically student projects don't reflect lessons learned in the real world
very well.


> Mr. Wallace (Don't have your first name),
> 
> Absolutely I agree, you're dead on target.  I think the Avalon stuff
> started off as Silk if I remember correctly.  But it moved into Excalibur
> and became the event project under the Excalibur umbrella.

Something along those lines.

> If I had my way I would implement SEDA classes as POJOs within an
> API and borrow from whatever I can in commons.  James Strachan has
> a nice little thread pool so I'd use that.  There are queues in the
> collections packages and you can decorate them with the ability to
> consult enqueue predicates but I think someone must have already beat
> us to it.  I would then use these vanilla classes with the various 
> IoC containers frameworks to create container specific wrappers 
> around the Stage POJO implementation.  This way you can remain container
> independent and reuse as much tested tried and true commons code as 
> possible.  

Feel free to try.  The Event package was created because the original
offering was too convoluted to really incorporate into your own work.


-- 

"They that give up essential liberty to obtain a little temporary safety
  deserve neither liberty nor safety."
                 - Benjamin Franklin


Mime
View raw message