httpd-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From gord...@cygnus.com (Gordon Irlam)
Subject Re: AddModule patch
Date Fri, 20 Sep 1996 03:33:32 GMT
Jim Jagielski <jim@jaguNET.com>:
> I like the concept of these patches, but I (being devil's advocate)
> have to question how much they would really be used. The main
> advantage I would see, as has been mentioned, would be for the
> precompiled binaries. However, I doubt if the serious webmaster
> would keep the "bloated" binary, but would eventually simply
> recompile with whatever modules needed to reduce the memory
> footprint.

I think even the serious webmaster might decide to keep the bloated
binaries.

Here is an analysis of the relative cost of running a web server that
is maxed out with the standard modules pre-compiled in, versus
re-compiling it with just the modules you need compiled in.

Option 1: Cost of running the "bloated binary"

I have a Linux Apache binary handy (it has all the core modules, and a
few of the extension modules configured in).  I am using perhaps 15
modules.  The size of it is pretty small.

    text    data    bss
    135k     29k    18k    httpd

Now, lets get nasty, and imagine building a really huge, maxed out
version of Apache with every module imagineable.  I'll really go
overboard here.  Let's add Perl, and Python, and Tcl, and whatever
else we might be able to find.  The Python binary I have is rather
old, and it includes Tk for what it is worth, but what the heck:

    text    data    bss
    410k     86k     5k    perl5
    962k     82k    13k    python
    244k     54k     2k    tclsh

And lets imagine -- I don't know where we would find this many -- but
lets imagine adding 100 more other regular Apache modules.  I'll
produce an overestimate for the space taken up by these by multiplying
the size of the entire original httpd by 5.

    text    data    bss
    675k    145k    90k    100 additional apache modules

Adding this all up, gives:

    text    data    bss
   2426k    396k   128k    httpd_bigmf

That's one mean Apache binary.  But how big is this really?  Note,
that it is still a lot smaller than my Netscape browser binary.  And
most people are able to afford to spend far more in terms of disk
space and memory to run a web server than they can a browser.

Imagine we have 20 server processes running.  Let's looking more
closely at the costs...

(I most familiar with Solaris, but all modern operating systems are
pretty similar.)

Swap space is reserved for each of these processes at fork time.  Text
however is paged from the executable, and so only the data, bss, and
heap allocated space really matter.  If the modules aren't being used
we aren't using any extra heap data (well, perhaps 10k for the extra
tables listing the unused modules, but that is all), we are using an
extra 370k for data, and 110k for bss.  In total this adds up to an
extra 500k of swap space per process, or with 20 processes, 10M of
pre-allocated swap space in total.

The processes will also be using up extra physical memory.  When httpd
is first executed the entire executable will be preloaded into memory,
but then over time unused pages are going to get paged out, and only
the working set of each process will remain in memory.  If none of the
additional modules are in active use, they effectively won't be
occupying any physical memory.  A slight correction to this however
needs to be made.  Previously all 15 of the modules I had been using
were packed together.  In the new binary they will be spread out, and
there will be on average an extra 1/2 page before and after the text,
data, and bss, of each module with code or data belonging to some
other unused module.  This consumes (1/2 + 1/2) * 3 * 15, or roughly
50 pages of memory per process.  Page sizes today are typically 4k.
With 4k pages this means 200k of memory wasted per process, plus
another 12k for the tables listing unused modules.  With 20 processes,
this adds up to 4M of RAM.

(The above 4M figure is something of an overestimate, since not all of
the original 15 modules I mentioned are likely to be in active use.
In reality, I am probably not using some of these 15 modules regularly
or at all.  Each such module would be responsible for both wasting an
equivalent of 3 pages in the old binary next to its neigbouring
modules, and in addition will not be wasting any space itself in the
new binary, which translates into double savings.  With 20 processes,
2-3M of extra memory is probably a more accurate estimate).

The total cost of running this monster version of Apache is an extra
4M of RAM and 10M of swap space.  Street prices for memory and disk in
US currency are currently 5 dollars per Meg, and 10 cents per Gig
(both have really fallen sharply recently; 16M for $75 and a 2.5G IDE
disk for $245).  Thus the cost is $20 for RAM and $1 for disk.

Thus the total extra cost of running this huge version of Apache, that
includes Perl, Tcl, Python, and 100 other Apache modules, when running
with 20 concurrent server processes, is only an extra $21.

Option 2: Cost of rebuiding the binary

The other option, instead of running a bloated binary, is to rebuild
it, with just the modules being used.  This involves someone figuring
out what modules they need, configure them, building the binary,
install it, and restarting the server.

A competant sysadmin might be able to do all this in perhaps 1 hour
assuming they have all the tools they need and nothing goes wrong.  In
the real world sometimes things go wrong, and so it is likely to take
much longer.  (Cygnus might typically budget 1-2 days to do something
like this for a customer).

With overheads, a cheap, competant sysadmin might cost $50/hr, and so
the cost to rebuild is $50.  If the site ever changes it's mind as to
what modules it wants to use this cost will need bearing again.



I hope this explains why being able to configure modules without
rebuild makes sense for a lot of people, and why Cygnus would like to
compile in all the modules people might want to use in the version of
Apache we ship.  Not everyone might agree this is the right thing, and
so Tom's changes don't require that all modules be pre-compiled in,
they merely enable it.

                                           regards,
                                                    gordoni

Mime
View raw message