Return-Path: Delivered-To: apmail-directory-dev-archive@www.apache.org Received: (qmail 17445 invoked from network); 24 Mar 2005 16:55:55 -0000 Received: from hermes.apache.org (HELO mail.apache.org) (209.237.227.199) by minotaur.apache.org with SMTP; 24 Mar 2005 16:55:55 -0000 Received: (qmail 79357 invoked by uid 500); 24 Mar 2005 16:55:54 -0000 Delivered-To: apmail-directory-dev-archive@directory.apache.org Received: (qmail 79216 invoked by uid 500); 24 Mar 2005 16:55:54 -0000 Mailing-List: contact dev-help@directory.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: "Apache Directory Developers List" Delivered-To: mailing list dev@directory.apache.org Received: (qmail 79202 invoked by uid 99); 24 Mar 2005 16:55:53 -0000 X-ASF-Spam-Status: No, hits=0.5 required=10.0 tests=FROM_ENDS_IN_NUMS,SPF_HELO_PASS,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (hermes.apache.org: domain of aok123@bellsouth.net designates 205.152.59.64 as permitted sender) Received: from imf16aec.mail.bellsouth.net (HELO imf16aec.mail.bellsouth.net) (205.152.59.64) by apache.org (qpsmtpd/0.28) with ESMTP; Thu, 24 Mar 2005 08:55:52 -0800 Received: from [172.16.1.7] ([65.80.200.112]) by imf16aec.mail.bellsouth.net (InterMail vM.5.01.06.11 201-253-122-130-111-20040605) with ESMTP id <20050324165550.ZTWJ1995.imf16aec.mail.bellsouth.net@[172.16.1.7]> for ; Thu, 24 Mar 2005 11:55:50 -0500 Message-ID: <4242F116.2090506@bellsouth.net> Date: Thu, 24 Mar 2005 11:55:50 -0500 From: Alex Karasulu User-Agent: Mozilla Thunderbird 1.0 (X11/20041206) X-Accept-Language: en-us, en MIME-Version: 1.0 To: Apache Directory Developers List Subject: Re: [mina] Performance issues References: <768dcb2e05032207031c81d58@mail.gmail.com> <9252c5466b3478355c434129e6393583@minq.se> <768dcb2e050323065960073866@mail.gmail.com> <187f93fcf0941f531163f22a9bfd9226@minq.se> <4242C8BD.5020607@d-haven.org> In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Virus-Checked: Checked X-Spam-Rating: minotaur.apache.org 1.6.2 0/1000/N Vinod Panicker wrote: >On Thu, 24 Mar 2005 09:03:41 -0500, Berin Loritsch > wrote: >-- snip -- > > > >The real fun will be coming when I start with the throughput tests. >That will definitely show the real power (or lack thereof) of NIO. >Which gets me to the question, how can throughput be effectively >measured? > > Yes that will be fun. Well throughput will be application specific based on the nature of the server. You're testing obviously is centered around MINA and NIO limits. This boils down to how much data the server over all can consume and how much it can pump out as we vary the number of concurrent connections. (1) Avoid tests with servers that access disk Minimizing the number of unrelated variables are key, like for example disk access. I'd recommend testing the echo server which does not hit disk like say an LDAP server would. (2) Turn off swap I would also making sure swapping does not come into the picture. The tests should be designed not to consume physical memory. Perhaps even disabling swap might be good idea. Just have the tested apps stop when the machine runs out of physical memory so you don't get tainted data with swapping. (3) Disable as man applications and services as possible Prevent resource utilization by other running processes. You want to minimize the overall CPU and memory taken up by other processes. Try printing out stats to a buffer instead of a log file and dumping them at the end of the test. (4) Measure throughput as data in/out per second (or even per 100 millisecond) samples per avg number of connections There are a couple options here. You can use Ethereal to collect the information or the server itself with filters to gather your stats into a buffer. Again the stat buffer should be dumped to a file at the end of the test (perhaps in a shutdown hook in case you get out of memory JVM crashes). This however might not give you the kind of resolution you're looking for. I would have a MINA filter measure per second samples on the amount of data going in and coming out. Within MINA this can be correlated with the number of average concurrent connections for each sample. These samples can then be used to construct the throughput graph where total IO per sample is ploted against the avg concurrent connections per sample on the x-axis. >Guys, repeating the call for suggestions for the benchmark suite for >MINA to make it as real-world a test as possible. > > > (5) [optional] Use a good bell curve distribution for message sizes It will be fun to generate several throughput curves using constant message sizes. This might tell us more about how we can deal with buffer sizes within MINA. This is not very realistic but helps understand some sweet spots in the MINA implementation. After you run a few tests using constant message sizes to compare the resultant curves it might be neat to see how things change when the message sizes are varied with a realistic distribution. A series of plots can be generated where the size distribution's maxima is displaced around the buffer size. This may tell us more about real world dynamics. Final Notes These code (which hopefully will become tools) for capturing throughput will be very useful for analyzing the nature of different servers and configurations. I think server implementors using MINA will certainly benefit from them rather than tuning their servers in the dark. This is really great work you're doing Vinod. Hope some of these suggestions help and that you're not dismayed by the amount of traffic - in particular around your research :). We're all very interested in your results. Thanks, -- Alex