directory-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From joshua portway <joshli...@stain.org>
Subject Re: [mina] controlling packets
Date Mon, 23 Jan 2006 14:54:58 GMT
thanks for the excellent help Trustin, I'll try removing all the  
packetizing stuff from the encoders themselves, and putting it in an  
IO filter as you suggest - it sounds like a much better place for  
that stuff to be. It never seemed quite right putting it in the  
encoder, but I've only been looking at Mina for a couple of days, so  
I don't think I've quite got the hang of the architecture yet.

One thing I'm not sure I've understood - In your PacketisingFilter  
implementation, it looks as if the encoder will still be having to  
split the message data into buffer size chunks before sending it  
"down the pipe" to the Packetizing filter which seems to be just  
adding the header to each buffer. It would seem cleaner for the  
encoder to just create the encoded message as one big ByteBuffer and  
the PacketisingFilter to split this up into packets and add the  
header to each packet.  If I did try to do all the packetizing in the  
filter, though, I'm still not sure how I would cause a "flush"  
downstream from the filter to actually write the header and data to  
the network as a single packet. Also, if I did write the IO filter  
like this, and there's a method to cause a "flush", does the method  
block until the data has actually been sent, or am I still going to  
have to deal with the problem of not being able to slice the buffer ?


Is there any movement on this bug yet - it sounds like this may be  
the next issue I have :
http://issues.apache.org/jira/browse/DIRMINA-162
I'd be willing to help with a solution if I can.

j

On 23 Jan 2006, at 14:40, Trustin Lee wrote:

> Hi Joshua,
>
> 2006/1/23, joshua portway <joshlists@stain.org>:
> Hi,
>         I have a protocol which communicates via UDP with a  
> hardware device
> which requires each message to bit split into separate network
> packets, each of which is preceded by a header which contains
> information about which message the packet is part of, what packet
> number it is, the total length of data in the message, and the
> position of the data in this packet in the message etc.
>
>         I'm not sure of the best way to implement this in Mina. I've
> implemented a custom ProtocolCodecFactory, loosely based on the Mina
> DemuxingProtocolCodecFactory, along with a MessageEncoder base class.
> Specific MessageEncoder subclasses write a series of ByteBuffers
> which make up th econtent of the message to a method in the baseclass
> and then call a "flush" method which divides the data up into
> packets, prepends the message header to the packet and sends it to
> the ProtocolEncoderOutput.
>
> It sounds like you did a great job!
>
>         The problem I'm getting is that if the message I'm sending  
> has data
> contained in a big ByteBuffer (which most of my messages do), then I
> need to divide the buffer into specific packet sized chunks to send.
> Since Mina ByteBuffers don't have a "split" method (presumably
> because they get re-used) I have been setting the position and limit
> of the input buffer to point to the chunk I want to send, and then
> passing it off to the ProtocolEncoderOutput. The problem is that
> because the ProtocolEncoderOutput runs on another thread, I can't
> just change the buffer position and limit for the next chunk and
> send it because I have to wait for the ProtocolEncoderOutput to
> finish before I change the ByteBuffer positions. The only other
> alternative would be to copy each packet into a new buffer which
> seems quite wasteful. Latency and throughput are VERY important to my
> app, because it's encoding video streams and pushing them to the
> network in real time, so I'd rather not be doing a lot of copying of
> buffers.
>
>         My current home-made network layer doesn't copy  
> ByteBuffers, but
> creates view buffers onto the source buffer using ByteBuffer.split,
> and passes the header for a packet and the view buffer for the data
> to the channels gathering write(ByteBuffer[]) method. Am I doing
> something wrong in the way I'm using Mina ? Is there a way to do this
> without copying buffers ?
>
> I understand your point completely.  It's also an issue for other  
> MINA users including ourselves.  There's already the JIRA issue  
> which is related with this problem:
>
> http://issues.apache.org/jira/browse/DIRMINA-165
>
> Please watch the issue, we'll keep updating our idea.  Of course,  
> you could post your idea, too. :)
>
>         Ideally, I think there should be a "packetizing" API in  
> Mina to
> allow easy control of how a message is packetized, and to allow each
> packet to be wrapped in a protocol-specific packet header. As far as
> I can tell no such thing exists, and the closes I've found is the
> "mergeAll" method of ProtocolEncoderOutput, but again- this seems to
> be implemented by copying all the input ByteBuffers into a single
> output ByteBuffer, which can't be the most optimal way of doing it,
> surely ? Or am I being naiive ? does the GatheringByteChannel.write
> (ByteBuffer[]) method just copy all it's input buffers into a single
> output buffer anyway ?
>
> This is a good idea.  Actually, you can implement it as an  
> IoFilter.  You could implement IoFilter.filterWrite() method to  
> write the header part:
>
> public class PacketizingFilter extends IoFilterAdapter {
>     public void filterWrite( NextFilter next, WriteRequest req ) {
>         ByteBuffer buf = ByteBuffer.allocate( ... );
>         // fill header here
>         buf.put( ... ) ....
>         ...
>         buf.flip ();
>         // Write header first
>         next.filterWrite( new WriteRequest( buf ) );
>         // and then body
>         next.filterWrite( req );
>     }
> }
>
> In your encoder, you don't need to call mergeAll in this case.   
> Please just call write()s.  The data will be flushed when encode()  
> method returns.  So it's basically similar with write(bb[]) unless  
> you called flush() manually.
>
>         thanks for any help, sorry about the long length of the  
> email :)
>
> It was a lot of fun to know that MINA is going to be used to  
> implement real-time media streamer! :)
>
> HTH,
> Trustin
> -- 
> what we call human nature is actually human habit
> --
> http://gleamynode.net/
> PGP Key ID: 0x854B996C


Mime
View raw message