httpd-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From (Wan-Teh Chang)
Subject Re: NSPR (was Re: rewritelog inefficiency)
Date Tue, 28 Apr 1998 22:26:23 GMT

Dean Gaudet wrote:

> Right, the PRFileDesc I'm using for logs will have no layers.  But I'm
> unclear on why the atomicity is not guaranteed for sockets and pipes
> when using a native-threads model (i.e. pthreads).

In NSPR, every fd that represents a socket or pipe on Unix is put innonblocking
mode (O_NONBLOCK flag is on).

> If NSPR is built using pthreads only, then isn't PR_Write() a glorified
> wrapper around write()?  i.e. it goes down the layers, the lowest layer
> is just a blocking write() because pthreads are taking care of the
> multithreading for you.  So you'll have the same atomicity guarantee as
> write() does, regardless of whether the PRFileDesc is a file, socket,
> or pipe.

If a PRFileDesc is in blocking mode, which is the default (note: this blocking
modeis at the NSPR level, not at the Unix level.  The Unix fd is always
PR_Write() does not return until all the data is written.  So it may have to
several write() system calls, as follows:
    write()    /* get EAGAIN, or a byte count less than you requested */
    write()    /* get EAGAIN, or a byte count less than you requested */
    write()   /* finally the entire user buffer has been transmitted */

You can see this write-poll loop in FileWrite() in prio.c and SocketWrite() in
prsocket.c and pt_Write() and pt_write_cont in ptio.c.

So, even if the write() system call is atomic (are you sure this
is true, by the way?), PR_Write() may not be atomic if a write() call
only writes part of the buffer.

> It'll be really unfortunate if we have to add locking for logs at the
> application layer.  In a pure pthreads setting, for example, we could
> just write() directly and it would all be taken care of by the kernel --
> the kernel has to lock anyhow, so it's not needed at the user level.

You are assuming that every write() will write the entire buffer.Suppose you
want to write 64K to a nonblocking socket,
write() may return 32K.  Then this write() is not atomic.
In the case of logging, you can probably assume that the data
buffer is less than 16K or 32K so that each write() will be able
to write the entire buffer when the fd is nonblocking.  If this
assumption is valid, then PR_Write() will only make one write()
system call, and therefore PR_Write() is also atomic.  But I don't
know if you are able to guarantee that all your write() calls are

> A related topic:  how well does a user-level threaded NSPR application
> deal with NFS filesystems?  I don't recall if O_NONBLOCK does
> anything useful in these cases.  In any event you can't do the open()
> asynchronously unless the kernel supports asynchronous i/o (like POSIX
> aio)... so it's mostly a moot point... if the NFS filesystem decides to
> lock up, the entire webserver will be stuck in an open().  The only way
> I see around this is to use native threads or aio.

We are not doing anything about opening a file across NFS, so theproblems you
cited will happen.


View raw message