httpd-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Chuck Murcko <>
Subject Re: WWW Form Bug Report: "sockets left in FIN_WAIT_2 lingering state" on SunOS 4.x (fwd)
Date Sat, 21 Dec 1996 23:44:02 GMT
Marc Slemko liltingly intones:
> On Thu, 19 Dec 1996, Chuck Murcko wrote:
> > # Network options.  NMBCLUSTERS defines the number of mbufs and defaults to 
> > # 256.  This machine is a gateway that handles lots of traffic, so we boost
> > # that value.
> > options         SOMAXCONN=256           # max pending connects
> Note that, unlike in BSD/OS, in FreeBSD SOMAXCONN is not wrapped by an
> ifndef, so defining it in your kenel config file won't work very well.  To
> modify it on FreeBSD the best way would be to use sysctl to modify
> "kern.somaxconn" after boot.  OTOH, it defaults to 128 in recent version
> of FreeBSD which is far more than BSD/OS defaulted to (16) last I checked.

Yes, it's risen to 64 now, but that's sometimes still not enough.

> > options         NMBCLUSTERS=4096        # clusters to spare(maybe)!
> When I last checked BSD/OS, it hardcoded NMBCLUSTERS.  FreeBSD does it
> based on maxusers using:
> 	int     nmbclusters = 512 + MAXUSERS * 16;

Hardcoded meaning you can change it from the config, but not on a running
kernel, right?

With either BSD/OS or FreeBSD you can force the NMBCLUSTERS value.
But you left out the lines that show this. You can do it either way:

>From FreeBSD param.c:

int     nmbclusters = 512 + MAXUSERS * 16;
int     nmbclusters = NMBCLUSTERS;

BSD/OS does its soft limit differently:

int     nmbclusters = NMBCLUSTERS;      /* current/"soft" limit */
int     maxmbclusters = MAXMBCLUSTERS;  /* hard limit */

from param.c. BSD/OS dynamically allocates MBCLUSTERS if you don't define
MAXMBCLUSTERS or NMBCLUSTERS. That is, it picks a value based on RAM in
the machine. It doesn't grow or shrink the allocated space dynamically
while running. (max 4096 unless overridden).

> which is reasonable in many cases.
> > 
> > Size of an mbuf_cluster in 4.4lite BSD is 16 kB. So, it's a compromise
> > as to where you want to set that value. Remember, you're sacrificing
> > RAM that could be running user procs by bumping it up.
> I thought mbuf clusters were around 1 or 2 Kbytes on most architectures?

Oops. Right. I just checked. 2k seems it on the FreeBSD here (2.1.5) and 
BSDI 2.1.
> FreeBSD's sys/i386/include/param.h:
>  #ifndef MCLSHIFT
>  #define MCLSHIFT        11              /* convert bytes to m_buf clusters */
>  #endif  /* MCLSHIFT */
>  #define MCLBYTES        (1 << MCLSHIFT) /* size of an m_buf cluster */
>  #define MCLOFSET        (MCLBYTES - 1)  /* offset within an m_buf cluster */
> which puts it at 2 kB, and lite2 uses:
>  #define MCLBYTES        1024
>  #define MCLSHIFT        10
> which is 1kB.
> We run lots of BSD servers with NMBCLUSTERS at 4096, and it doesn't take 
> 65536 kB of memory for the mbuf clusters.
Yep. I don't remember now where I looked to arrive at the original value.
It was probably a year ago. Thanks for catching this. I may have mistakenly
read a kmem cluster size or size of a list.

It also strengthens my suspicion that the limiting factor on how many
FIN_WAIT_2 connects a machine can tolerate is driven more by hitting the
limit of the list used to maintain them than by lossage of RAM. We're
definitely not out of RAM when our BSDI boxes panic.

As always (and as the maxuser line in my original mail stated) look
at the /usr/src/sys/params.c file for the formulae that will apply to
your version of 4.4lite.

Chuck Murcko	N2K Inc.	Wayne PA
And now, on a lighter note:
The light at the end of the tunnel may be an oncoming dragon.

View raw message