directory-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Trustin Lee <>
Subject Re: [mina] controlling packets
Date Mon, 23 Jan 2006 13:40:25 GMT
Hi Joshua,

2006/1/23, joshua portway <>:
> Hi,
>         I have a protocol which communicates via UDP with a hardware
> device
> which requires each message to bit split into separate network
> packets, each of which is preceded by a header which contains
> information about which message the packet is part of, what packet
> number it is, the total length of data in the message, and the
> position of the data in this packet in the message etc.
>         I'm not sure of the best way to implement this in Mina. I've
> implemented a custom ProtocolCodecFactory, loosely based on the Mina
> DemuxingProtocolCodecFactory, along with a MessageEncoder base class.
> Specific MessageEncoder subclasses write a series of ByteBuffers
> which make up th econtent of the message to a method in the baseclass
> and then call a "flush" method which divides the data up into
> packets, prepends the message header to the packet and sends it to
> the ProtocolEncoderOutput.

It sounds like you did a great job!

        The problem I'm getting is that if the message I'm sending has data
> contained in a big ByteBuffer (which most of my messages do), then I
> need to divide the buffer into specific packet sized chunks to send.
> Since Mina ByteBuffers don't have a "split" method (presumably
> because they get re-used) I have been setting the position and limit
> of the input buffer to point to the chunk I want to send, and then
> passing it off to the ProtocolEncoderOutput. The problem is that
> because the ProtocolEncoderOutput runs on another thread, I can't
> just change the buffer position and limit for the next chunk and
> send it because I have to wait for the ProtocolEncoderOutput to
> finish before I change the ByteBuffer positions. The only other
> alternative would be to copy each packet into a new buffer which
> seems quite wasteful. Latency and throughput are VERY important to my
> app, because it's encoding video streams and pushing them to the
> network in real time, so I'd rather not be doing a lot of copying of
> buffers.
>         My current home-made network layer doesn't copy ByteBuffers, but
> creates view buffers onto the source buffer using ByteBuffer.split,
> and passes the header for a packet and the view buffer for the data
> to the channels gathering write(ByteBuffer[]) method. Am I doing
> something wrong in the way I'm using Mina ? Is there a way to do this
> without copying buffers ?

I understand your point completely.  It's also an issue for other MINA users
including ourselves.  There's already the JIRA issue which is related with
this problem:

Please watch the issue, we'll keep updating our idea.  Of course, you could
post your idea, too. :)

        Ideally, I think there should be a "packetizing" API in Mina to
> allow easy control of how a message is packetized, and to allow each
> packet to be wrapped in a protocol-specific packet header. As far as
> I can tell no such thing exists, and the closes I've found is the
> "mergeAll" method of ProtocolEncoderOutput, but again- this seems to
> be implemented by copying all the input ByteBuffers into a single
> output ByteBuffer, which can't be the most optimal way of doing it,
> surely ? Or am I being naiive ? does the GatheringByteChannel.write
> (ByteBuffer[]) method just copy all it's input buffers into a single
> output buffer anyway ?

This is a good idea.  Actually, you can implement it as an IoFilter.  You
could implement IoFilter.filterWrite() method to write the header part:

public class PacketizingFilter extends IoFilterAdapter {
    public void filterWrite( NextFilter next, WriteRequest req ) {
        ByteBuffer buf = ByteBuffer.allocate( ... );
        // fill header here
        buf.put( ... ) ....
        // Write header first
        next.filterWrite( new WriteRequest( buf ) );
        // and then body
        next.filterWrite( req );

In your encoder, you don't need to call mergeAll in this case.  Please just
call write()s.  The data will be flushed when encode() method returns.  So
it's basically similar with write(bb[]) unless you called flush() manually.

        thanks for any help, sorry about the long length of the email :)

It was a lot of fun to know that MINA is going to be used to implement
real-time media streamer! :)

what we call human nature is actually human habit
PGP Key ID: 0x854B996C

View raw message