httpd-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Aaron Bannert <>
Subject Re: load spikes revisited
Date Thu, 17 Jan 2002 16:04:32 GMT
On Thu, Jan 17, 2002 at 10:20:00AM -0500, Greg Ames wrote:
> As a matter of fact, I typically listen on two ports while testing to disable
> S_L_U_A, so I can easily figure out which process will get the next connection
> in case I want to gdb it.  While trying out ktrace on my test config, I saw that
> the fcntl() accept mutex has got a thundering herd problem on daedalus.  After
> releasing the fcntl mutex, you see the kernel context switching to all of the
> idle httpd processes.  The first process that wakes up gets the mutex, the rest
> of the context switches simply burn a little CPU, then block again.  Moral:  the
> default cure looks as bad as the disease.

Is there anywhere else that we've started using cross process locks
since 2.0.28? If fcntl() is known to cause this behaviour, why is it
enabled at all on this version of FreeBSD?

Based on your ktrace output from a couple days ago, I have a working
theory that I have yet to reproduce: I noticed that there is a very
high occurance of sendfile returning with errno 35 (Resource temporarily
unavailable). The sendfile(2) man page reveals that this means sendfile
was called in non-blocking mode, and the kernel buffers have filled
up. A quick glance at sendfile_it_all in server/core.c looks to me
like it would loop repeatedly on that error condition, as we are seeing
it in the ktrace output. This means that on low-bandwidth connections
like modems, we'll burn cycles testing if the kernel buffer is ready
for more data while the modem still hasn't pulled the next packet down.
I haven't been able to reproduce this, since I am not exactly sure what
conditions are required to trigger sendfile. Can someone give me a
URL that will consistently trigger the use of sendfile?

Unfortunately, I don't think this will account for the short bursts of run
queue growth we're talking about here, but it is something to look into.


View raw message