Return-Path: 
 <dev-return-19342-apmail-httpd-dev-archive=httpd.apache.org@httpd.apache.org>
Delivered-To: apmail-httpd-dev-archive@httpd.apache.org
Received: (qmail 3984 invoked by uid 500); 23 Aug 2001 16:02:11 -0000
Mailing-List: contact dev-help@httpd.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@httpd.apache.org
list-help: <mailto:dev-help@httpd.apache.org>
list-unsubscribe: <mailto:dev-unsubscribe@httpd.apache.org>
list-post: <mailto:dev@httpd.apache.org>
Delivered-To: mailing list dev@httpd.apache.org
Received: (qmail 3971 invoked from network); 23 Aug 2001 16:02:11 -0000
Subject: Re: Flood
From: Ian Holsman <ianh@cnet.com>
To: dev@httpd.apache.org
In-Reply-To: <20010823085620.J17570@ebuilt.com>
References: <A5374D237E78D41195810090279CC91A021EE04D@xcup04.cup.hp.com>
	<20010816101517.O1397@ebuilt.com>
	<20010823155314.G33621@deejai2.mch.fsc.net>
	<20010823085620.J17570@ebuilt.com>
Content-Type: text/plain
Content-Transfer-Encoding: 7bit
X-Mailer: Evolution/0.12.99+cvs.2001.08.21.23.41 (Preview Release)
Date: 23 Aug 2001 09:01:16 -0700
Message-Id: <998582506.1490.4.camel@griffon.cnet.com>
Mime-Version: 1.0
X-Spam-Rating: h31.sny.collab.net 1.6.2 0/1000/N
Status: O
X-Status: 
X-Keywords: 
X-UID: 1099

Is it worth putting up a page somewhere on httpd.apache.org mentioning
the test stuff (with links to join etc, and a description of what is in
there ?)


On Thu, 2001-08-23 at 08:56, Justin Erenkrantz wrote:
> On Thu, Aug 23, 2001 at 03:53:14PM +0200, Martin Kraemer wrote:
> > > 
> > > You can devise a user profile that flood will execute (i.e. a sequence
> > > of URLs - potentially dynamic) rather than a single static URL as ab
> > > lets you do.  -- justin
> > 
> > How does it compare to httperf?
> 
> Well, Aaron, Roy, and myself did look at httperf before we started 
> development on flood.  However, you don't have to write C modules
> to extend flood's "workload generators" (to use httperf's jargon).  
> You can write C modules to extend how flood operates, but that's not
> required to specify which URLs you wish to hit.
> 
> Basically, we have the ability to mimic a complete user profile in a XML
> format.  This snippet is based off of examples/round-robin-example.xml
> in httpd-test (slightly cleaned up):
> 
>  <urllist>
>     <name>Apachelabs.org search profile</name>
>     <description>Search for httpd-2.0 commit access and get the first two messages in the thread</description>
>     <url method="POST" payload="method=and&amp;format=builtin-long&amp;sort=score&amp;config=htdig&amp;restrict=&amp;exclude=&amp;words=httpd-2.0+commit+access"
> responsetemplate="&lt;a href=&quot;([^&quot;]*)&quot;&gt;" responsename="id">http://www.apachelabs.org/cgi-bin/htsearch</url>
>     <url requesttemplate="${id}" responsetemplate="&lt;A HREF=&quot;([^&quot;]*)&quot;&gt;Next&lt;/A&gt;" responsename="next" />
>     <url requesttemplate="http://www.apachelabs.org${next}" />
>   </urllist>
> 
> The first URL does a POST as a htdig query searching for "httpd-2.0
> commit access" on apachelabs.org.  Notice the HTML escaping present -
> since we are using XML format, we must escape everything - that kind of
> sucks, but without writing our own pseudo-XML parser (we use APR-util's 
> XML API) we don't have much of a choice here.  =)
> 
> When the search returns, we look for the first occurrance of "<a
> href="http://www.apachelabs.org/foo/bar.html"> (regexp matching) and 
> place http://www.apachelabs.org/foo/bar.html in the variable id.  
> 
> The next URL simply uses that variable directly to be the next URL to
> hit.  Then, once we retrieve that URL, we look for 
> "<A HREF="/spaz/bar.html">Next</A> and store /spaz/bar.html into the 
> variable next.
> 
> We then hit the final URL in our sequence with that message.
> 
> See if you can do this in three lines with httperf.  =)
> 
> Current features that httperf supports that we don't:
> Chunked encoding (on our short list when we get around to it)
> Log-file replay (this is conceptually easy except for POST payloads)
> As much network information detail
> 
> Flood's output is pretty raw at this point.  We output it so that it is
> easy to run your own statistical analysis/summary on the results.  So,
> we expect you to write awk/perl scripts - we have a bunch that we use
> internally to generate result summaries.  I may commit modified versions
> of some of them when time permits.  (We also run flood in combination
> with sar and snoop at times to generate more OS specific data...)
> 
> Features we support that httperf doesn't:
> SSL
> Better delay handling (per URL)
> slightly-easier configuration
> Threading
> 
> Threading is a mixed bag performance-wise as poll() based is slightly 
> better performance wise, but remember that we aren't going for 
> meaningless benchmarks *choke* ab *choke*, but "real-world" numbers - 
> so a thread/user makes a lot of sense implementation-wise.
> 
> Like httperf, we'd like to have a "daemon-mode" where we can spread out
> flood sessions across multiple machines and have all of the reports go
> to one central place.
> 
> Anyway, we're using flood internally at eBuilt to performance test 
> one of our client's websites (the XML gets quite hairy as the site is 
> very complex).  So, we're trying to first build in the features
> that we need and then focus on other things later as time permits.
> We've found a number of bottlenecks and problematic pages under load
> on that site.  We're able to tell them which pages are worse under
> load, etc, etc.  So, it is scratching our itch.
> 
> Ideally, other people will find this tool of value and use it.  And,
> if they find problems (or need new features), they can submit patches
> (or commit themselves).  We'd definitely welcome the help.  HTH.  
> -- justin
-- 
Ian Holsman          IanH@cnet.com
Performance Measurement & Analysis
CNET Networks   -   (415) 364-8608