perl-modperl mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jeremy Howard" <jh_li...@fastmail.fm>
Subject Report on mod_accel and mod_deflate
Date Thu, 20 Dec 2001 02:43:17 GMT
Igor Sysoev mentioned recently on this list that he has written a module
called 'mod_accel' that provides a caching HTTP accelerator, as well as a
mod_gzip replacement called 'mod_deflate'. These modules are both used on
Kaspersky labs' busy sites, as well as at the popular portal
http://www.rambler.ru. mod_deflate is used at around 700 sites.

So why haven't we heard more of them? Because the docs are only in Russian,
and almost all implementations have occured on Russian sites. Well, the good
news is that I've now implemented a mod_accel/mod_deflate front-end on
http://www.fastmail.fm, allowing me to announce a substantial speedup to our
users:
  http://www.emailaddresses.com/forum/showthread.php?threadid=1603

You may notice in the linked announcement that I mention "I've been able to
remove the restriction that attachments have to be downloaded within an
hour". We previously had this restriction because even with mod_proxy as a
front-end, uploads larger than the TCP buffer get fed through bit by bit,
keeping the mod_perl handler open the whole time... So we had code like:
----
||   my $oldalarm=alarm(0);
||   alarm 3600;
||   my $Status = $R->parse;
||   alarm $oldalarm;
----
Without that, the backend app server can hang around for ever serving one
client. We had similar code for downloads, as well as uploads. BTW, Joe
Schaefer has posted a mod_proxy patch to this list that fixes this for POST
uploads.

Well, mod_accel gets around this problem by automatically buffering large
uploads and downloads onto the filesystem! :-) In fact, mod_accel reads the
backend response and sends it to client asynchronously, so the client sees
the first part of the response even before the backend has completed the
request. But the backend is never blocked by the client, since it is
buffered to the file-system as required.

mod_deflate works really well with mod_accel so that both dynamic forwarded
requests and static requests are compressed. mod_deflate also has lots of
intelligence to avoid compressing content to buggy browsers that can't
handle it.

OK, enough evangelism. On to practical matters. Many of the following
snippets and comments were provided to me by Igor in private email, so thank
him, not me. Firstly, here's how to compile everything (including mod_ssl on
the front-end) and install it into /usr/local/apache_accel:

----
tar -zxvf mod_accel-1.0.7.tar.gz
tar -zxvf mod_deflate-1.0.9.tar.gz
tar -zxvf mm-1.1.3.tar.gz
cd mm-1.1.3
./configure --disable-shared
make

cd ../mod_ssl-2.8.5-1.3.22
./configure --with-apache=../apache_1.3.22_accel/ \
  --with-crt=/INSERT_APPROPRIATE_PATH.crt \
  --with-key=/INSERT_APPROPRIATE_PATH.key

cd ../mod_accel-1.0.7
./configure --with-apache=../apache_1.3.22_accel/ \
  --with-eapi=../mod_ssl-2.8.5-1.3.22/pkg.eapi/
make

cd ../mod_deflate-1.0.9
./configure --with-apache=../apache_1.3.22_accel/
make

cd ../apache_1.3.22_accel/
EAPI_MM=../mm-1.1.3 SSLBASE=SYSTEM ./configure --enable-rule=EAPI \
  --prefix=/usr/local/apache_accel --disable-rule=EXPAT \
  --enable-module=most \
  --enable-module=rewrite \
  --activate-module=src/modules/accel/libaccel.a \
  --activate-module=src/modules/extra/mod_deflate.o
make
make install

mkdir /var/accelcache
chown nobody:nobody /var/accelcache/
/usr/local/apache_accel/bin/apachectl startssl
----

Now, in /usr/local/apache_accel/conf/httpd.conf, append the following:
----
   AccelCacheRoot  /var/accelcache 1
   AccelNoCache    on
   AccelPass    /mail/        http://127.0.0.1:8080/mail/
   AccelWaitBeforeBodyRead  100
   AccelUnlinkNoCached  off
   AccelSetXHost on
   AccelSetXRealIP on
   AccelSetXURL on
----

This creates an HTTP accelerator without any caching. I don't know much
about the caching directives and haven't tried them out, so I can't provide
any guidance there.

The AccelCacheRoot directive sets the number of levels of subdirectories in
the cache. 16 first level subdirectories named 0 .. f and 256 second level
subdirectories named 00 .. ff are used by default (i.e. "AccelCacheRoot
<path> 1 2"). Since I'm not caching, my config above does not specify any
directory hashing (i.e. "AccelCacheRoot <path> 1").

AccelNoCache simply turns off caching.

AccelPass specifies that I want everything under /mail/ forwarded to the
backend. You can also specify '/' for AccelPass, and then use something like
"AccelNoPass  /download  /images  ~*\.jpg$" to create exceptions (uses
standard regex syntax).

Note that mod_accel can also be called by utilising the mod_rewrite [P]
directive, just like with mod_proxy.

AccelWaitBeforeBodyRead sets the time in milliseconds to wait between
receiving the response header and the first part of the body. It is useful
over Ethernet where packets are small. If mod_accel waits a little it can
read most of the body or even the whole body with one just one
select()/read() pair of syscalls. You can see the first read size and the
number of reads in the log with %{accel}x in a CustomLog directive. Igor
usually uses 100-200 milliseconds.


AccelSet* adds X-* headers to the request to the backend. This is useful to
know what the original request details were.

Now, some directives that I've left at the default in my config...
AccelSendSize sets the buffer size when sending the answer to the client.
AccelBkRcvSockBuffSize sets the kernel backend receive buffer size. It's the
same as mod_proxy's ProxyReceiveBufferSize. AccelBkRcvBuffSize sets the
mod_accel backend receive buffer size. If the response size is more then
this, then it is saved to a temporary file. AccelRequestBuffSize sets
mod_accel's client receive buffer size when receiving the
POST body. If the request body is more then this then it is saved to a
temporary file.

You can leave default values for all these buffers and increase
AccelBkRcvBuffSize and AccelRequestBuffSize only if you see many warnings in
the log. Igor suggests not setting these buffers to more then 1M. The
directives are all specified in kB (quick Russian lesson:
kB=='kilobajtah'!).

Here's the config for mod_deflate:
----
   DeflateEnable On
   DeflateMinLength 3000
   DeflateCompLevel 1
   DeflateProxied Off
   DeflateHTTP 1.0
   DeflateDisableRange "MSIE 4."
----

DeflateProxied Off means that anything with a 'Via' header is not
proxied--Igor says that some proxies are broken so this is best to be safe.
And MSIE 4 apparently is a bit broken too. By default only text/html is
compressed. DeflateCompLevel 1 provides adequate compression for text.



Mime
View raw message