directory-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Berin Loritsch <>
Subject Re: Proposing some changes to the ProtocolProvider API
Date Fri, 10 Dec 2004 14:13:16 GMT
Alex Karasulu wrote:


> Sorry but from a first look that's a negative.  All your response 
> objects are expected to be created in advance.  Namely I'm referring 
> to he Object[] returned from the processRequest() method.  Basically 
> think about Search operations in Eve....
> Search requests can return 1 or more responses that equals zero or 
> more ExtendedResponses or SearchResultEntryResponses followed by as 
> SearchResultDone to terminate the request.  Eve is designed to stream 
> out search results rather than keep an entire result set in memory.  
> By having an Object[] return all responses at once I have to cache all 
> the results before even starting to return any entries.  This will 
> make Eve extremely inefficient.  Hence the reason why I return an 
> Iterator.  Every time you ask for the next response its pulling an 
> entry from the database which matches the filter.
> Eve does not fetch an entry until you're ready to return it to a 
> client.  This keeps the perclient resources used to a minimum. Without 
> this performance would degrade considerably.  So look at using a 
> Cursor friendly (one that can wrap a cursor) construct like Iterators, 
> Enumerations, et, cetera.

I can understand where you are coming from.  However, the current 
protocol provider solution is far too complex to really digest 
quickly--hense making a simple Echo server or something of that nature 
is too complicated.  I boiled everything down as much as possible to 
start the conversation going.

I believe that something needs to change--and I am not necessarily 
looking for change *before* release.  I think that both of the proposed 
network layers are slated more for the next release of Eve.  However, 
the details of how things change are always up for discussion.

Based on this information, I would like to propose a modification to 
what I originally put up.  More below.

>> This will allow the smallest footprint for really small and easy 
>> systems, yet allows
>> more complex protocol providers to delegate to helper objects as need 
>> be.  I have
>> no problems with using the current factory approach either--as long 
>> as the encode/
>> decode process returns something that can be passed on.
> I like the factory approach because you can delay when the 
> encoder/decoder pairs are created and by which module.  There is lots 
> of flexibility here.   Otherwise you're basically making the provider 
> the codec as well as the request processor.  These are three separate 
> functions that need not be coupled.  Let's explore why the 
> ProtocolProvider interface is so inhibitive.  What problems are you 
> having with it?

The problems I am having with it are as follows:

1) Part of the API is in Snickers while the other part is in Seda--tying 
two implementations together
2) More complex than it needs to be--leading to more time trying to 
figure the thing out
3) It is too wed to the current API to be retrofit easily into what 
either Trustin or I are doing
4) Encoders do not encode and respond... they encode and it goes into an 
5) Same with Decoders...

The thing is that in order to implement a Protocol you need no less than 
6 objects--even if your needs are very minimal.  I boiled everything 
down to one interface--recognizing that internal to that interface some 
protocols might want to use separate incoders/decoders/ etc.  However, 
since that is shifting the bulk of the work on the protocol writer, I 
can temper the request a bit.

>> Currently the encoder explicitly publishes the "encode event" 
>> directly to the
>> event publisher from within client code.  In this sense it is 
>> explicitly tied to the
>> current SEDA implementation.
> There is an EncoderManager which does the publishing since as a 
> component it holds a dep on the event router.  No encoder will publish 
> to an event queue.  Encoders simply trigger the manager to enqueue.

Yes and this manager you speak of is, again, an extra step that might 
not be needed.  It is an added complexity that we can most likely do 

>> How does this sound?
> Sorry Berin I cannot go with this for the reasons stated inline.  
> However I'm open to working with you.  I just need to know why you're 
> having problems.  I also want to make sure you're familiar enough with 
> the current SEDA and Eve code to understand why I made most of my 
> design choices and why I cannot veer away from certain mechanisms.
> Alex

Currently, the ProtocolProvider has three methods: getDecoderFactory(), 
getEncoderFactory(), and getRequestHandler()

That is fairly workable; however, there are two levels of 
abstraction--the ProtocolProvider is a factory for the request handler, 
but only returns the factory for the encoders and decoders.  Can we 
simply make the ProtocolProvider a Factory?  That would simplify it in 
this way:

// Responsibility of ProtocolProvider to write
interface ProtocolProvider
    StatefulDecoder getDecoder( Socket client, DecoderCallBack callback );

    RequestHandler getRequestHandler( Socket client, RequestCallBack 
callback );

    StatefulEncoder getEncoder( Socket client, EncoderCallBack callback );

// Responsibility of ProtocolProvider to write
interface StatefulDecoder
    // internally calls the DecoderCallBack when something is there to 
send on
    void decode(ByteBuffer buffer);

// Responsibility of ProtocolProvider to write
interface RequestHandler
    // internally calls the RequestCallBack when any response is ready
    void handleRequest(Object fromDecoder);

// Responsibility of ProtocolProvider to write
interface StatefulEncoder
    // internally calls the EncoderCallBack when buffers are ready.
    void encode(Object fromRequestHandler);

// Responsibility of network system to write
interface DecoderCallBack
    void decoded(Socket client, Object requestObject);

// Responsibility of network system to write
interface RequestCallBack
    void response(Socket client, Object responseObject);

// Responsibility of network system to write
interface EncoderCallBack
    void encoded(Socket client, ByteBuffer buffer);

This takes things like the "XXXManager" out of the picture--the 
responsibilties are much smaller.
It also makes it easier to forward events asynchronously when they 
happen.  As far as the network
is concerned, only the Socket identifies a client--session objects and 
other more complex things are
part of whatever protocol is being serviced by the network layer.

And more importantly, this allows us to identify a specific 
ProtocolProvider API.  This API can
(and should) be a dependency of the network layers and the individual 
providers.  However the
protocol providers should not have to have a dependency on the network 

Currently we have part of the provider API provided by snickers, and yet 
another part of it provided
by the network API.  It makes the project hierarchy more difficult to 
manage as well.  By separating
out the protocol provider API into its own _clearly_marked_ subproject, 
we can clean up the
dependency structure in a way that truly decouples the network layer 
from the protocol layer--which
also gives us more of an opportunity to switch things up and optimize 
things without fearing breaking
the different protocol providers.

It also makes it tremendously easier to add new protocol providers as 


"Programming today is a race between software engineers striving to build bigger and better
idiot-proof programs, and the Universe trying to produce bigger and better idiots. So far,
the Universe is winning."
                - Rich Cook

View raw message