httpd-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Dean Gaudet <>
Subject NSPR (was Re: rewritelog inefficiency)
Date Tue, 28 Apr 1998 20:57:34 GMT

On Tue, 28 Apr 1998, Wan-Teh Chang wrote:

> Hi, I am in the NSPR group.  You are right, NSPR does not guaranteethat
> writes to a PRFileDesc are atomic.  So if you have several threads
> writing to the the same PRFileDesc (as is the case in logging), it's safer
> to
> lock the fd when you write to ensure atomicity.

Hi!  Nice to see you on the list.

> If the write system calls of the underlaying OS are atomic, and the
> PRFileDesc in question is not layered (e.g., no buffering layer),
> then very likely the NSPR writes to *normal files* are also atomic.
> (NSPR turns on the O_NONBLOCK flag for sockets and pipes
> on Unix, but does not do so for normal (disk) files.)
> But I'll need to examine our source code carefully to be sure.

Ok I was hoping to walk through the code before posting, but I'll be way
busy today and tomorrow.  Maybe my misunderstanding of NSPR's low layers
is completely off balance, not unlikely given that I've only spent two
weeks with it.

Right, the PRFileDesc I'm using for logs will have no layers.  But I'm
unclear on why the atomicity is not guaranteed for sockets and pipes
when using a native-threads model (i.e. pthreads).

If NSPR is built using pthreads only, then isn't PR_Write() a glorified
wrapper around write()?  i.e. it goes down the layers, the lowest layer
is just a blocking write() because pthreads are taking care of the
multithreading for you.  So you'll have the same atomicity guarantee as
write() does, regardless of whether the PRFileDesc is a file, socket,
or pipe.

If NSPR is built using your user-level threads library, doesn't a
filesystem PR_Write() just push down into a blocking write() as well?
So again you'll have the same atomicity guarantee as write() does --
for files.

For pipes and sockets, with the user-level threads, you must have a queue
somewhere deep down, probably below the lowest layer.  You need to have
a list of what each thread is waiting for so that you can construct the
poll() event list.  So you've got some form of locking already down there
(you're only locking out your scheduling interrupt I'd guess).  Given that
lock already exists, it's overkill for us to lock at the top level.

The combined MxN model should be the same as the user-level threads model.

But... I'm probably missing something.  I'm probably ignoring something
in the PRFileDesc layers themselves -- like the need to protect concurrent
access to each layer.

It'll be really unfortunate if we have to add locking for logs at the
application layer.  In a pure pthreads setting, for example, we could
just write() directly and it would all be taken care of by the kernel --
the kernel has to lock anyhow, so it's not needed at the user level.

When I get a chance I'm going to single-step through the library and watch
it do all this stuff... then I'll probably have a better feel for it.

A related topic:  how well does a user-level threaded NSPR application
deal with NFS filesystems?  I don't recall if O_NONBLOCK does
anything useful in these cases.  In any event you can't do the open()
asynchronously unless the kernel supports asynchronous i/o (like POSIX
aio)... so it's mostly a moot point... if the NFS filesystem decides to
lock up, the entire webserver will be stuck in an open().  The only way
I see around this is to use native threads or aio.

It'll be fun trying to explain to users all the different choices of
processing models they'll have available and what each is good at/bad
at :)


View raw message