httpd-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Graham Dumpleton <>
Subject Re: Problems with EOS optimisation in ap_core_output_filter() and file buckets.
Date Tue, 17 Feb 2009 06:25:49 GMT
2009/2/17 Joe Orton <>:
> On Mon, Feb 16, 2009 at 10:52:15PM +1100, Graham Dumpleton wrote:
>> 2009/2/16 Joe Orton <>:
>> > You say:
>> >
>> >> For me this is an issue as the file descriptor has been supplied from
>> >> a special object returned by a higher level application and it would
>> >> be hard to maintain the file as open beyond the life of the request,
>> >> up till end of keep alive or a subsequent request over same
>> >> connection. Doing a dup on the file decriptor is also not necessarily
>> >> an option.
>> >
>> > can you explain why a dup shouldn't work?
>> I did used to perform a dup, but was told that this would cause
>> problems with file locking. Specifically was told:
> I'm getting lost here.  What has file locking got to do with it?  Does
> mod_wscgi rely on file locking somehow?

The mod_wsgi package is a gateway for hosting Python web applications.
Modern version of mod_python, but implementing generic/portable Python
WGSI interface rather than being Apache specific. It is what all the
Python people ditching mod_python are moving to.

In WSGI applications, it is possible for the higher level Python web
application to pass back a file object reference for the response with
the intent that the WSGI adapter use any optimised methods available
for sending it back as response. This is where file buckets come into
the picture to begin with.

Now, this file object was created by the Python application and it is
still the owner of it. If it is a file that it had first been
modifying and it needed exclusivity on that, it could well have used
file locks on it. Because locks are involved, the order in which files
contents are used for response and the closure and unlocking of the
file are important.

It appears that fcntl locking on some platforms has the behaviour that
if file descriptor is dup'd, closure of the first reference to the
file will cause release of the lock. That is, lock will not be
released only when last reference to the file is closed. Problems
therefore can arise if you have to dup the file descriptor, because if
the dup'd file descriptor gets closed before Python application had
finished with the file object, possibly involving it having to modify
the file after contents sent, something else could lock it and start
modifying the file before it was done.

In simple terms, the mod_wsgi module doesn't internally open the file
in the first place. Instead it comes from a higher level application
and one can't do things at the lower level that would change the state
of something like the locks associated with the file.


View raw message