httpd-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From dean gaudet <dgaudet-list-new-ht...@arctic.org>
Subject Re: mod_file_cache performance
Date Tue, 03 Jul 2001 00:15:00 GMT
On Mon, 2 Jul 2001, Cliff Woolley wrote:

> On Mon, 2 Jul 2001, Cliff Woolley wrote:
>
> > For keepalive requests, since this particular file is less than
> > AP_MIN_BYTES_TO_WRITE in size, the core_output_filter will read in the
> > file and hang on to it to try to send at the same time as the response to
> > the next request using writev().  That's the whole bug we were trying to
> > solve with this latest round of patches to mod_file_cache.
>
> I just realized I'm making assumptions that people know how the file
> buckets read code works.  By "read in the file", I meant "read from the
> file bucket", which since on my system APR_HAS_MMAP and the file is of an
> appropriate size means that the file would be MMAPed and then passed to
> writev() a few responses at a time.  On systems without APR_HAS_MMAP or in
> certain other edge situations, "read in the file" would really mean "read
> the file into a buffer on the heap", but not in this case.

oh that's bad.

mmap() does not scale in a threaded program on a multi-cpu system -- each
time you change the memory mapping you essentially have to issue an
interrupt to all CPUs possibly running threads, or with possibly cached
TLB mappings in the same memory space (which essentially means all the
CPUs in the box, no matter what architecture you consider).

mmap() is less painful in a multiprocess server on a machine without
context ids in the TLB... which describes x86 SMP boxes.  (some CPUs, such
as the usparc, have "context ids" in the TLB which allow you to load up
entries from multiple vm maps at the same time, and keep them in the TLB
across task switches, so you have to non-local flush TLBs on a memory map
change even if there's only one task with that memory map.  the x86
doesn't have this, and a task switch includes a TLB flush of user pages,
so you never have any non-local mappings for an unshared memory map.)

sendfile() is the best thing to use on linux (and probably everywhere else
that has it).  it doesn't require any virt->phys mappings to be created,
and supports zero-copy.

a mostly optimal syscall sequence for responses to a keep-alive
connection on linux should look something like:

	sockfd = accept();
	fcntl(sockfd, F_SETFL, O_NDELAY)
	setsockopt(sockfd, TCP_CORK = 1)
	while (1) {
		rc = read(sockfd);
		if (rc <= 0) {
			save_errno = errno;
			/* send any remaining packets now */
			setsockopt(sockfd, TCP_CORK = 0);
			if (rc == 0) break;
			if (save_errno == EAGAIN) {
				poll(until we can read sockfd);
				continue;
			}
			/* log error */
			break;
		}
		/* parse request */
		respfd = open(response_file);
		write(sockfd, response headers)
		sendfile(sockfd, respfd)
		close(respfd)
	}
	close(sockfd);

obviously with suitable mods for doing timeouts and crud like that.

wasn't this how apache 2.0 worked on linux prior to the filters/buckets?

-dean

p.s. i say "mostly" optimal because poll() is not the best choice, the
signal events are much better.  see <http://www.kegel.com/c10k.html> and
read up on the "realtime signal" stuff.


Mime
View raw message