From olio-user-return-74-apmail-incubator-olio-user-archive=incubator.apache.org@incubator.apache.org Mon Feb 09 04:43:27 2009 Return-Path: Delivered-To: apmail-incubator-olio-user-archive@minotaur.apache.org Received: (qmail 81024 invoked from network); 9 Feb 2009 04:43:26 -0000 Received: from hermes.apache.org (HELO mail.apache.org) (140.211.11.2) by minotaur.apache.org with SMTP; 9 Feb 2009 04:43:26 -0000 Received: (qmail 80816 invoked by uid 500); 9 Feb 2009 04:43:26 -0000 Delivered-To: apmail-incubator-olio-user-archive@incubator.apache.org Received: (qmail 80805 invoked by uid 500); 9 Feb 2009 04:43:25 -0000 Mailing-List: contact olio-user-help@incubator.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: olio-user@incubator.apache.org Delivered-To: mailing list olio-user@incubator.apache.org Received: (qmail 80796 invoked by uid 99); 9 Feb 2009 04:43:25 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Sun, 08 Feb 2009 20:43:25 -0800 X-ASF-Spam-Status: No, hits=0.2 required=10.0 tests=SPF_PASS,URIBL_GREY X-Spam-Check-By: apache.org Received-SPF: pass (athena.apache.org: domain of williamvoor@gmail.com designates 209.85.198.226 as permitted sender) Received: from [209.85.198.226] (HELO rv-out-0506.google.com) (209.85.198.226) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 09 Feb 2009 04:43:16 +0000 Received: by rv-out-0506.google.com with SMTP id g9so1310382rvb.4 for ; Sun, 08 Feb 2009 20:42:56 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:in-reply-to:references :date:message-id:subject:from:to:content-type :content-transfer-encoding; bh=ffHCtvbMtT5VNonpKHv9BGSciuS02ikvVK3CGDEZYTU=; b=PRg0O1yENz3mqnI2VnN1n/dzHtnj4IsrnZmfgNhs+bjxIT62vAsJukjwKexhyR2KzY FBcV/WjLEBtCbapuJXdJ5hz0+SAcx5lhgk2LihiXKZfmiIu/ry+xoId532ZAOyrnB1lc ZJ1HnZCnIPLer6mr7dcPJ2E6ezqKKlkakPX3g= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type:content-transfer-encoding; b=YMHIsJ5kPYOINkmnXntcj+qVrFl7YMteGuZXzPp4w25b7m4O7kOVkc35ctix+g5Ka0 bQrVl8kaUPqB1IPQ5iniNdDELixk6DLo0FzLxzndvlLx9CZbMF0Jsb2unQ8YRjHCvYOZ 8iqMXrKyqg7eNc38UanlpTAF6ZQ3eQtErAkSw= MIME-Version: 1.0 Received: by 10.141.19.9 with SMTP id w9mr3478298rvi.273.1234154575215; Sun, 08 Feb 2009 20:42:55 -0800 (PST) In-Reply-To: <498CDF39.3080701@sun.com> References: <4264c9500902021829k17e6df93jf42c6663823374c0@mail.gmail.com> <498882AD.9080408@sun.com> <4264c9500902041917i43ee49bck49039a42c5fd7e7c@mail.gmail.com> <498B55AC.7040500@sun.com> <4264c9500902051532j68e354c6o68bea12876fc8668@mail.gmail.com> <498CDF39.3080701@sun.com> Date: Mon, 9 Feb 2009 15:42:55 +1100 Message-ID: <4264c9500902082042s7b04a105n271e3162e29f04bf@mail.gmail.com> Subject: Re: Benchmark ideal runtime? From: William Voorsluys To: olio-user@incubator.apache.org Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit X-Virus-Checked: Checked by ClamAV on apache.org Hello Shanti, I guess there's a problem with the patches. It looks like they are inverted. In any case, I presumed --- means +++ and patched the actual files in web20php2/public_html. But nothing changed in my results. A sharp decline in throughput still takes place after 20 minutes or so, mainly regarding Add Event. I'm still running a version downloaded in early January. Was there any significant bug fix that might solve this problem? If yes, I can try upgrading the whole thing. Cheers, William! On Sat, Feb 7, 2009 at 12:09 PM, Shanti Subramanyam wrote: > We just discovered a bug because of which the system does more and more work > as time goes by. The effect is usually not noticeable until you get to quite > a large scale, run for a very long time or are pressed for cpu. I'm not sure > if this is the issue you're running into as I don't see an increase in cpu > time on your webserver as time goes by, but your response time graph looks > exactly like what we were seeing. > > I'm attaching patches for two files in web20php2/public_html > (addAttendee.php and events.php) - can you please try these and let us know > if the problem persists ? > > Shanti > > William Voorsluys wrote: >> >> Hello >> >> On Fri, Feb 6, 2009 at 8:10 AM, Akara Sucharitakul >> wrote: >> >>> >>> Hi William, >>> >>> What is the number of attendees added during the steady state? >>> >> >> That would be 64888 * 0.0948 = 6151 >> >> EventDetail (success count) 64888 >> % EventDetail views where attendee added 9.48 >> >> Complete report is attached! >> >> >>> >>> In the summary report, under miscellaneous stats, there's the "% >>> EventDetail >>> views where attendee added" field. Use this percentage point and >>> calculate >>> the added attendees from the success count of EventDetail (first table - >>> "Operation Mix.") >>> >>> Thanks, >>> -Akara >>> >>> William Voorsluys wrote: >>> >>>> >>>> Hello, >>>> >>>> Here are some details about a 1 hour run that shows when the bursts >>>> happen. >>>> The tools I ran were "mpstat -P ALL 5; vmstat 5; iostat -x 5". >>>> However, in this for some reason the Detail Results page was not >>>> generated for this run. Thus, the graph attached belongs to another >>>> run, which was identical but different tools were running. >>>> System 'william2' is the web server (Ubuntu server 8.04 64-bit, 2 >>>> CPUs, 3GB RAM). >>>> System 'mysql1' runs only the database (Debian Etch 32-bit, 4 CPUS, 1GB >>>> RAM). >>>> Both the filestore and the database files are on NFS. >>>> >>>> Cheers, >>>> >>>> William >>>> >>>> >>>> >>>> On Wed, Feb 4, 2009 at 4:45 AM, Shanti Subramanyam >>>> wrote: >>>> >>>>> >>>>> I have noticed some bursts at 15-20 mins, but I think these are >>>>> specific >>>>> to >>>>> our deployment (nfs issues) and we're working on tuning those - we >>>>> haven't >>>>> identified any Olio issue that will cause this behavior (at least not >>>>> yet >>>>> !) >>>>> I assume you're running with a local filestore ? I'd be happy to take a >>>>> look >>>>> at one of your run outputs (assuming you collect vmstat, iostat etc.) >>>>> >>>>> Shanti >>>>> >>>>> William Voorsluys wrote: >>>>> >>>>>> >>>>>> Hello, >>>>>> >>>>>> What is the minimum time I should run Olio's PHP workload to be sure >>>>>> my system really supports a certain amount of concurrent users? I've >>>>>> been running for 30 minutes on steady state and all metrics pass, but >>>>>> extending the run to 1 hour make things fail. It seems that at 25 >>>>>> minutes there's a workload burst that destabilize my system. >>>>>> Is there a documentation that describes workload characteristics, such >>>>>> as when bursts take place? >>>>>> >>>>>> Thanks, >>>>>> >>>>>> William >>>>>> >>>>>> >>>> >>>> >>>> ------------------------------------------------------------------------ >>>> >>>> >>>> System parameters on server william2 >>>> >>>> # >>>> # /etc/sysctl.conf - Configuration file for setting system variables >>>> # See sysctl.conf (5) for information. >>>> # >>>> >>>> #kernel.domainname = example.com >>>> >>>> # the following stops low-level messages on console >>>> kernel.printk = 4 4 1 7 >>>> >>>> # enable /proc/$pid/maps privacy so that memory relocations are not >>>> # visible to other users. (Added in kernel 2.6.22.) >>>> kernel.maps_protect = 1 >>>> >>>> # Increase inotify availability >>>> fs.inotify.max_user_watches = 524288 >>>> >>>> # protect bottom 64k of memory from mmap to prevent NULL-dereference >>>> # attacks against potential future kernel security vulnerabilities. >>>> # (Added in kernel 2.6.23.) >>>> vm.mmap_min_addr = 65536 >>>> >>>> ##############################################################3 >>>> # Functions previously found in netbase >>>> # >>>> >>>> # Comment the next two lines to disable Spoof protection (reverse-path >>>> filter) >>>> # Turn on Source Address Verification in all interfaces to >>>> # prevent some spoofing attacks >>>> net.ipv4.conf.default.rp_filter=1 >>>> net.ipv4.conf.all.rp_filter=1 >>>> >>>> # Uncomment the next line to enable TCP/IP SYN cookies >>>> # This disables TCP Window Scaling (http://lkml.org/lkml/2008/2/5/167) >>>> #net.ipv4.tcp_syncookies=1 >>>> >>>> # Uncomment the next line to enable packet forwarding for IPv4 >>>> #net.ipv4.ip_forward=1 >>>> >>>> # Uncomment the next line to enable packet forwarding for IPv6 >>>> #net.ipv6.ip_forward=1 >>>> >>>> >>>> ################################################################### >>>> # Additional settings - these settings can improve the network >>>> # security of the host and prevent against some network attacks >>>> # including spoofing attacks and man in the middle attacks through >>>> # redirection. Some network environments, however, require that these >>>> # settings are disabled so review and enable them as needed. >>>> # >>>> # Ignore ICMP broadcasts >>>> #net/ipv4/icmp_echo_ignore_broadcasts = 1 >>>> # >>>> # Ignore bogus ICMP errors >>>> #net/ipv4/icmp_ignore_bogus_error_responses = 1 >>>> # # Do not accept ICMP redirects (prevent MITM attacks) >>>> #net/ipv4/conf/all/accept_redirects = 0 >>>> # _or_ >>>> # Accept ICMP redirects only for gateways listed in our default >>>> # gateway list (enabled by default) >>>> # net/ipv4/conf/all/secure_redirects = 1 >>>> # >>>> # Do not send ICMP redirects (we are not a router) >>>> #net/ipv4/conf/all/send_redirects = 0 >>>> # >>>> # Do not accept IP source route packets (we are not a router) >>>> #net/ipv4/conf/all/accept_source_route = 0 >>>> # >>>> # Log Martian Packets >>>> #net/ipv4/conf/all/log_martians = 1 >>>> # >>>> # Always defragment packets >>>> #net/ipv4/ip_always_defrag = 1 >>>> >>>> >>>> >>>> Processor info for server william2 >>>> >>>> >>>> >>>> processor : 0 >>>> cpu family : 6 >>>> model : 23 >>>> model name : Intel(R) Xeon(R) CPU E5410 @ 2.33GHz >>>> cpu MHz : 2327.498 >>>> cache size : 6144 KB >>>> cpu cores : 1 >>>> cpuid level : 10 >>>> cache_alignment : 64 >>>> processor : 0 >>>> cpu family : 6 >>>> model : 23 >>>> model name : Intel(R) Xeon(R) CPU E5410 @ 2.33GHz >>>> cpu MHz : 2327.498 >>>> cache size : 6144 KB >>>> cpu cores : 1 >>>> cpuid level : 10 >>>> cache_alignment : 64 >>>> >>>> >>>> >>>> Memory info for server william2 >>>> >>>> >>>> >>>> MemTotal: 3139292 kB >>>> MemFree: 1072572 kB >>>> SwapCached: 0 kB >>>> SwapTotal: 409616 kB >>>> SwapFree: 409548 kB >>>> >>>> >>>> >>>> Kernel on server william2 >>>> >>>> >>>> >>>> Linux 2.6.24-23-xen #1 SMP Thu Nov 27 20:14:09 UTC 2008 x86_64 >>>> >>>> >>>> >>>> ------------------------------------------------------------------------ >>>> >>>> >>>> ------------------------------------------------------------------------ >>>> >>>> >>>> >>>> >>>> System parameters on server mysql1 >>>> >>>> >>>> >>>> # >>>> # /etc/sysctl.conf - Configuration file for setting system variables >>>> # See sysctl.conf (5) for information. >>>> # >>>> >>>> #kernel.domainname = example.com >>>> #net/ipv4/icmp_echo_ignore_broadcasts=1 >>>> >>>> # Uncomment the following to stop low-level messages on console >>>> #kernel.printk = 4 4 1 7 >>>> >>>> ##############################################################3 >>>> # Functions previously found in netbase >>>> # >>>> >>>> # Uncomment the next line to enable Spoof protection (reverse-path >>>> filter) >>>> #net.ipv4.conf.default.rp_filter=1 >>>> >>>> # Uncomment the next line to enable TCP/IP SYN cookies >>>> #net.ipv4.tcp_syncookies=1 >>>> >>>> # Uncomment the next line to enable packet forwarding for IPv4 >>>> #net.ipv4.conf.default.forwarding=1 >>>> >>>> # Uncomment the next line to enable packet forwarding for IPv6 >>>> #net.ipv6.conf.default.forwarding=1 >>>> >>>> >>>> >>>> Processor info for server mysql1 >>>> >>>> >>>> >>>> processor : 0 >>>> cpu family : 6 >>>> model : 23 >>>> model name : Intel(R) Xeon(R) CPU E5410 @ 2.33GHz >>>> cpu MHz : 2327.498 >>>> cache size : 6144 KB >>>> cpuid level : 10 >>>> processor : 1 >>>> cpu family : 6 >>>> model : 23 >>>> model name : Intel(R) Xeon(R) CPU E5410 @ 2.33GHz >>>> cpu MHz : 2327.498 >>>> cache size : 6144 KB >>>> cpuid level : 10 >>>> processor : 2 >>>> cpu family : 6 >>>> model : 23 >>>> model name : Intel(R) Xeon(R) CPU E5410 @ 2.33GHz >>>> cpu MHz : 2327.498 >>>> cache size : 6144 KB >>>> cpuid level : 10 >>>> processor : 3 >>>> cpu family : 6 >>>> model : 23 >>>> model name : Intel(R) Xeon(R) CPU E5410 @ 2.33GHz >>>> cpu MHz : 2327.498 >>>> cache size : 6144 KB >>>> cpuid level : 10 >>>> >>>> >>>> >>>> Memory info for server mysql1 >>>> >>>> >>>> >>>> MemTotal: 1048688 kB >>>> MemFree: 499280 kB >>>> SwapCached: 0 kB >>>> SwapTotal: 522072 kB >>>> SwapFree: 522072 kB >>>> >>>> >>>> >>>> Kernel on server mysql1 >>>> >>>> >>>> >>>> Linux 2.6.18.8.xs5.0.0.10.439 #1 SMP Wed Aug 6 23:55:12 UTC 2008 i686 >>>> >>>> >>>> >>> >>> >> >> >> >> >> ------------------------------------------------------------------------ >> >> >> Web20Bench 0.4 >> Summary Report >> >> >> Web20Bench metric: 73.112 ops/sec >> Benchmark Start: Thu Feb 05 10:54:18 EST 2009 >> Benchmark End: Thu Feb 05 12:04:21 EST 2009 >> Run ID: Web20Driver.36W >> Pass/Fail: FAILED >> Active Drivers: UIDriver <#UIDriver> >> >> >> ------------------------------------------------------------------------ >> >> >> UIDriver >> >> >> UIDriver metric: 73.112 ops/sec >> Driver start: Thu Feb 05 10:54:18 EST 2009 >> Driver end: Thu Feb 05 12:04:21 EST 2009 >> Total number of operations: 263202 >> Pass/Fail: FAILED >> >> >> Operation Mix >> >> Type Success >> Count Failure >> Count Mix Required Mix >> (2% deviation allowed) Pass/Fail >> HomePage 68576 0 26.05% 26.15% PASSED >> Login 26857 0 10.2% 10.22% PASSED >> TagSearch 88202 0 33.51% 33.45% PASSED >> EventDetail 64888 0 24.65% 24.68% PASSED >> PersonDetail 6896 0 2.62% 2.61% PASSED >> AddPerson 2189 0 .83% .84% PASSED >> AddEvent 5594 0 2.13% 2.04% PASSED >> >> >> Response Times (seconds) >> >> Type Avg Max 90th% Reqd. 90th% Pass/Fail >> HomePage 0.846 8.596 2.900 1.000 FAILED >> Login 0.859 7.883 2.960 1.000 FAILED >> TagSearch 0.306 7.896 1.120 2.000 PASSED >> EventDetail 0.233 3.729 0.880 2.000 PASSED >> PersonDetail 0.246 3.137 0.800 2.000 PASSED >> AddPerson 0.403 3.361 0.940 3.000 PASSED >> AddEvent 1.465 19.198 4.420 4.000 FAILED >> >> >> Cycle/Think Times (seconds) >> >> Type Targeted Avg Actual Avg Min Max Pass/Fail >> HomePage 4.961 4.955 0.000 25.094 PASSED >> Login 4.918 4.911 0.000 25.006 PASSED >> TagSearch 4.991 4.984 0.000 25.056 PASSED >> EventDetail 4.939 4.932 0.000 25.103 PASSED >> PersonDetail 4.971 4.964 0.000 25.004 PASSED >> AddPerson 4.737 4.730 0.000 25.001 PASSED >> AddEvent 4.915 4.908 0.000 25.025 PASSED >> >> >> Miscellaneous Statistics >> >> Description Results Targeted >> Results Allowed >> Deviation Pass/Fail >> % EventDetail views where attendee added 9.48 >= 6 >> PASSED >> EventDetail count where attendee can be added 60424 >> >> >> Average images references on Home Page 10.00 10 0.5 PASSED >> Average images loaded per Home Page 3.42 >= 3 >> PASSED >> Average image bytes received per Home Page 17777.55 >= 15000 >> >> PASSED >> Average images on Tag Search Results 4.19 >= 7 >> FAILED >> Average images on Event Detail 1.00 >= 1 >> PASSED >> Total successful AddEvent calls 6347 >> >> >> Total successful AddPerson calls 2493 >> >> >> >> >> Little's Law Verification >> >> Number of users = 400 >> Sum of Avg. RT * TPS for all Tx Types = 398.8362 >> >> >> ------------------------------------------------------------------------ > > --- addAttendee.php Thu Feb 5 13:08:52 2009 > +++ addAttendee.php.orig Thu Feb 5 13:08:41 2009 > @@ -43,10 +43,7 @@ > if (!isset($connection)) { // If connection not there, we're read-only. > $connection = DBConnection::getInstance(); > } > -$listquery = "select username from PERSON_SOCIALEVENT ". > - "where socialeventid = '$se' and username = '$username' ". > - "union select username from PERSON_SOCIALEVENT ". > - "where socialeventid = '$se' limit 20"; > +$listquery = "select username from PERSON_SOCIALEVENT where socialeventid = > '$se'"; > $listqueryresult = $connection->query($listquery); > $username = $_SESSION["uname"]; > while($listqueryresult->next()) { > > --- events.php Thu Feb 5 13:08:47 2009 > +++ events.php.orig Thu Feb 5 13:08:34 2009 > @@ -76,21 +76,11 @@ > $result->get(13).",".$result->get(14); > } > unset($result); > -if (isset($_SESSION["uname"])) { > - // Ensure our user name comes in first, if already attending. > - $listquery = "select username from PERSON_SOCIALEVENT ". > - "where socialeventid = '$se' and username = '$username' ". > - "union select username from PERSON_SOCIALEVENT ". > - "where socialeventid = '$se' limit 20"; > - > -} else { > - $listquery = "select username from PERSON_SOCIALEVENT ". > - "where socialeventid = '$se' limit 20"; > -} > +$listquery = "select username from PERSON_SOCIALEVENT where socialeventid = > '$se'"; > $listqueryresult = $connection->query($listquery); > while($listqueryresult->next()) { > $tmp_uname = $listqueryresult->get(1); > - if (!isset($_SESSION["uname"]) && $tmp_uname == $username) { > + if (!is_null($_SESSION["uname"]) && $tmp_uname == $username) { > $unattend = true; // show unattend button if user is already > registered. > } > $attendeeList = $attendeeList." ".' href="users.php?username='.$tmp_uname.'">'.$tmp_uname.'
'; > > -- William Voorsluys williamvoor.googlepages.com