directory-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Trustin Lee <trus...@gmail.com>
Subject Re: [mina] Performance issues
Date Thu, 24 Mar 2005 06:13:23 GMT
Hi Vinod,

On Thu, 24 Mar 2005 10:50:21 +0530, Vinod Panicker <vinod.p@gmail.com> wrote:
<snip/>

> Alright... Latest stats are as follows -
> 
> Server - 30K concurrent connections on a Linux box with 512 MB RAM.
> Ran out of Direct Buffer Memory (more on this later)
> Client - 30K concurrent connections from a single Windows 2003 server
> box 1 GB RAM (no problems)

Sounds great.

> Basically the client is going full throttle with connect requests.
> Obviously we will run benchmarks with more sedate clients that
> occasionally cause load spikes.
> 
> I agree with what Berin says, and every Internet server should know
> its limits.  Thats where these benchmarks help - we can estimate what
> kind of load is MAX, under which the system can still provide expected
> performance levels.

We'll be able to create a IoHandlerFilter that limits the maximum
number of connected clients.

> Alex, there's still lot more to do.  Will compile all of this into
> something that Trustin can put someplace.  I'm also tinkering around
> with JVMStat to obtain exact figures.  Regarding the data limits per
> partition, I wont be able to help out over there (I think you should
> be looking at external storage for such kind of benchmarks)

I think we can provide your testing code into separate subproject.

> Alright, problem areas -
> 
> 1. No backlog support currently provided with IoProtocolAcceptor

You're right.  I've just added SocketAcceptor.get/setBacklog() method.
 So you can:

((SocketAcceptor) ioProtocolAcceptor.getIoAcceptor()).setBacklog( yourBacklog );

> 2. When the server accepts a connection, 8K of Direct Buffer Memory is
> being allocated.  I suspect this is too much.  Depending on where MINA
> is used, this memory should be tunable.  From what I saw in the code,
> this is for the read buffer.  Taking an example of a Directory Server
> such as ApacheDS, I doubt incoming data is going to fill the 8K
> buffer.  And since its direct buffer, more care should be taken to use
> up this kind of memory.  If MINA is being used for an FTP server, this
> figure would make sense.  My suggestion would be to make the buffer
> size and kind of buffer (direct, non-direct) values tunable.

I reduced it to 1K and reconfigurable, and added
SocketSessionConfig.setSessionReceiveBufferSize(...). :)

> 3.  Client/Server CPU usage.  The number of connections per second
> steadily keeps decreasing as the total connections increase.  Client &
> Server both are at 100% CPU.  That is what is weird.  I'm aiming to
> isolate the problem.  I'm currently a bit doubtful about the way the
> Connector works, but wont be able to point out anything concrete until
> more permutations & combinations are tried.

Hmm, interesting.  This might be related with
processIdleSessions(...).  Could you try to disable it?

> I'm expecting that MINA should be able to take 100K connections on the
> server.  It cant go over 65K for the client due to lack of available
> port numbers.

Can we accept more than 64K connections on the server side?

<snip/>

Thanks for your consistent contribution! :)

Regards,
Trustin
-- 
what we call human nature is actually human habit
--
http://gleamynode.net/

Mime
View raw message