> > Ye gads.  Why can't you multithread the reads?  What possible reason
> > could one come up with that the reads have to happen on the same thread
> > as the select?
> 
> Because it doesn't work ;) 
> 
> I could offer a reasonable explanation of why that is so but ultimately it 

Could you give the longer explanation to me.  I just thought that if one thread in a stage detects the input PDU (Protocol Data Unit) another worker thread in a stage down stream could handle reading the PDU into a buffer for decoding.

> comes down to the fact that the underlying OSes are inconsistent - 
> particularly when there is multiple CPUs. The java layers on top proably 
> introduce more ways in which sligt differences can occur. 
> 
> Consider the scenario
> 
> CPU1 returns Select with FD3
> 
> > > See attached files for the sort of thing I mean. Associated with each
> > > connection object I have a TcpTransport object. The selector threads
> > > does all
> > > the nbio and then sends event to "parsing" stage. This seems to work
> > > like a
> > > charm.
> >
> > Don't that serialize the I/O?
> 
> Not sure what you mean. It is a little bit "unfair" because usually the 
> SelectorKeys are returned in order of underlying FDs and thus channels early 
> in the set are always early in the set. But as long as you randomizethe 
> processing order this should not be an issue.
> 
> As long as you dont do memory allocation in selector thread then all that 
> thread does is select() and memcpy() which can scale relatively well even if 
> you batch it and randomize events when you pass them o0nto next stage.
> 

Sounds like you think we're doing more than we are in the selector thread.  Basically a thread in the input detection stage has a loop where it blocks
waiting for input on socket channels.  When it wakesup due to incomming 
requests or a timeout it can hand off the reading to the worker threads of another stage.  I don't really see how OS specifics play here.  However it 
does sound like you read a very interesting artical on the disparity across
operating systems with resspect to nio, so give us a link ;-).

BTW here's an alternative (not so main stream option) that uses light weight
threads.  Basically a pool of java execution stacks share a single thread and 
in themselves act like mini threads.  Theres a PicoTHreads implementation out there that use byte code splicing to enable these Java continuations.  Here's the paper:

http://www.xcf.berkeley.edu/~jmacd/cs262.pdf

They also have non-blocking io primitives that do the same things the selectors and channels try to achieve.

Very interesting but I do not know if this is a good alternative to the selector based approach which has been a standard in C for well I can't remember how far back.  I used to use it when programing berkely sockets.

Alex