directory-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Wes McKean" <>
Subject Re: [asn.1] Decoder service design ( was: Paper on original SNACC)
Date Thu, 22 Jan 2004 17:11:40 GMT
TLV = Tag Length Value, the basic ASN.1 construct.  I'm edumacated :-)

I'm gonna read this in more detail later, but I wanted to respond right away to the question
of streams versus ByteBuffers...

What I did in my stuff, for testing, was create a sink.  The sink acted like an InputStream
for the thread reading the data, and it was a non-blocking repository for the thread doing
the reading.  In other words...

The reading thread reads byte buffer, finds the sink for the client connection, and ads the
byte buffer to its queue.  done.

The thread doing the decoding reads from the input stream, and blocks when it runs out of
data.  When the reader thread drops another byte buffer into the sink/queue, the decoder thread
takes off again.

This is a lot simpler than writing a decoder that works like a state machine.  It kind of
hides the source of the input, and allows the decoder to deal with a stream.

Not saying this is the way to go, but this was my idea from earlier...


---------- Original Message ----------------------------------
From: Alex Karasulu <>
Reply-To: "Apache Directory Developers List" <>
Date:  Thu, 22 Jan 2004 11:10:32 -0500

>> From: "Wes McKean" <>
>> I don't have permission to access the snacc.pdf.  What do I do?
>Sorry dude I just set the perms to allow access.  Give it another 
>> I'm going to go ahead and get started on the BER encoder/decoder.  For now,
>> I will keep the interface the same as the current one.  Namely, the
>> encoder/decoder will read from and write to a stream.  The primary
>> difference is I will detect non-blocking streams.  If anyone has any idea on
>> this, let me know.  It might truly be difficult to implement the decoder as
>> non-blocking, if that means decoding TLVs in chunks.  
>TLVs? Forgive me I'm bad with acronyms.  Oh and another thing take a look at
>the new front end stuff in eve/frontend/common/api where there are events 
>defined.  Namely take a look at the InputEvent which carries with it a 
>direct nio buffer.  Look towards defining your interfaces around this.
>> That would require the decoder to almost be like a state machine.  
>> If we decide that is the route to go, then fine with me.  For getting 
>> started, I will use just a regular old stream.
>I've given limited time to thinking about this one.  Basically we have
>chunks of data that will be delivered over to the decoder in the form
>of input events associated with a client.  The InputEvent (caught a bug)
>should extend the ClientEvent (which it currently does not - my bad!) 
>which has the ClientKey associated with it.  The InputEvent is a SEDA
>So what you do with the buffer content is upto you but here are some
>thoughts.  I'm thinking out loud here.  
>First of let me state a couple things about the existing state of 
>the frontend.  Don't worry about manipulating the byte buffer with 
>a concern for effecting other peices of code that work with the same 
>input event.  When you access the buffer you really have a read only 
>copy of the buffer which is backed by the direct buffer allocated 
>from the buffer pool.  Meaning you can advance and read and do a 
>bunch o stuff to change where you are in the buffer but you cannot 
>affect the content to mess with other code that may also have the 
>event delivered to it.  So advancing in the buffer will not throw of
>other listeners that have the event delivered to them.
>You got the buffer.  What you do with it is upto you.  
>Now the decoder service may eventually be used to decode more than
>just input comming in from a client stream down the road.  Just wanted
>to mention this but I don't think you should factor it into what you're
>dealing with right now.  This is for decoding stuff through the LdapContext
>JNDI interface which still use encoded ASN.1 for Controls.  I'll deal
>with this later but just thought I'd mention it.
>Ok we want to basically not block or rather dedicate a thread while 
>processing an entire message.  That would mean two threads used per
>request.  One thread to deliver the event, and another to read and 
>decode it.  So this is the main problem.
>Perhaps we can setup a small framework here inside the decoder with
>a callback.  When a new message is detected, a non-direct buffer can 
>be assigned to span the time required to decode the request.  This
>buffer collects the data coming in from the direct buffer in the
>InputEvent.  The input buffers are copied into the collector.  Now
>if the decoder was a state machine then we would not have to do a
>copy right?  The state machine based decoder could read from the 
>direct buffer and between chunks freeze its state until the next
>chunk appears.  That would be the best approach as you mentioned but
>I don't know how complex that would be.  And when processing a chunk
>the state machine should return even when it has not reached a 
>terminal state.
>Now if we continue on the less efficient collector buffer approach 
>then you need to have the decoder detect the start of a new PDU and
>the end of a PDU (Protocol Data Unit an ASN.1 message envelope).  It 
>must detect the start of the PDU to create and assign a collector 
>buffer to the current request.  It must detect the end of the PDU
>to basically trigger the true decoding machinery to slurp up the
>content in the collector buffer and decode it which can be done in
>one step and can be blocking.  You can use 1.4 regex functionality
>to detect the PDU start sequence and the end sequence of bytes 
>comming in as you transfer bytes from the direct buffer into the
>collector buffer.  Again this is sounding convoluted too.
>What are your thoughts?  There has got to be a better way perhaps
>we just need to sleep on it.

View raw message