httpd-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From <...@covalent.net>
Subject Re: mod_file_cache performance
Date Tue, 03 Jul 2001 02:19:09 GMT
On Mon, 2 Jul 2001, dean gaudet wrote:

> On Mon, 2 Jul 2001, Cliff Woolley wrote:
>
> > On Mon, 2 Jul 2001, Cliff Woolley wrote:
> >
> > > For keepalive requests, since this particular file is less than
> > > AP_MIN_BYTES_TO_WRITE in size, the core_output_filter will read in the
> > > file and hang on to it to try to send at the same time as the response to
> > > the next request using writev().  That's the whole bug we were trying to
> > > solve with this latest round of patches to mod_file_cache.
> >
> > I just realized I'm making assumptions that people know how the file
> > buckets read code works.  By "read in the file", I meant "read from the
> > file bucket", which since on my system APR_HAS_MMAP and the file is of an
> > appropriate size means that the file would be MMAPed and then passed to
> > writev() a few responses at a time.  On systems without APR_HAS_MMAP or in
> > certain other edge situations, "read in the file" would really mean "read
> > the file into a buffer on the heap", but not in this case.
>
> oh that's bad.
>
> mmap() does not scale in a threaded program on a multi-cpu system -- each
> time you change the memory mapping you essentially have to issue an
> interrupt to all CPUs possibly running threads, or with possibly cached
> TLB mappings in the same memory space (which essentially means all the
> CPUs in the box, no matter what architecture you consider).
>
> mmap() is less painful in a multiprocess server on a machine without
> context ids in the TLB... which describes x86 SMP boxes.  (some CPUs, such
> as the usparc, have "context ids" in the TLB which allow you to load up
> entries from multiple vm maps at the same time, and keep them in the TLB
> across task switches, so you have to non-local flush TLBs on a memory map
> change even if there's only one task with that memory map.  the x86
> doesn't have this, and a task switch includes a TLB flush of user pages,
> so you never have any non-local mappings for an unshared memory map.)
>
> sendfile() is the best thing to use on linux (and probably everywhere else
> that has it).  it doesn't require any virt->phys mappings to be created,
> and supports zero-copy.
>
> a mostly optimal syscall sequence for responses to a keep-alive
> connection on linux should look something like:
>
> 	sockfd = accept();
> 	fcntl(sockfd, F_SETFL, O_NDELAY)
> 	setsockopt(sockfd, TCP_CORK = 1)
> 	while (1) {
> 		rc = read(sockfd);
> 		if (rc <= 0) {
> 			save_errno = errno;
> 			/* send any remaining packets now */
> 			setsockopt(sockfd, TCP_CORK = 0);
> 			if (rc == 0) break;
> 			if (save_errno == EAGAIN) {
> 				poll(until we can read sockfd);
> 				continue;
> 			}
> 			/* log error */
> 			break;
> 		}
> 		/* parse request */
> 		respfd = open(response_file);
> 		write(sockfd, response headers)
> 		sendfile(sockfd, respfd)
> 		close(respfd)
> 	}
> 	close(sockfd);
>
> obviously with suitable mods for doing timeouts and crud like that.
>
> wasn't this how apache 2.0 worked on linux prior to the filters/buckets?

No.  It is however how Apache 2.0 works on linux with the filters/buckets
code, assuming no filters try to examine every character in the file.
Basically, if we have to read the data, then we have to MMAP.  If we don't
read from the file, then we just use sendfile.  So, assuming no SSI or
other parsing filter, we just use sendfile on linux.

Ryan

_____________________________________________________________________________
Ryan Bloom                        	rbb@apache.org
Covalent Technologies			rbb@covalent.net
-----------------------------------------------------------------------------


Mime
View raw message