httpd-apreq-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Markus Wichitill <ma...@gmx.de>
Subject Apache::Upload API
Date Thu, 17 Jun 2004 12:50:45 GMT
Hi,

there are two scenarios where the current upload API with buckets,
slurp() and link() doesn't perform too well (unless I'm missing
something).

One is the common case of simply storing an uploaded file. Under MP1,
this was easy, first I try to create another hard link() for the spool
file, and if that fails because the destination is on another
filesystem, I use the filename to make a copy.

With the MP2 version, if link() fails, I either have to slurp() the
whole file into memory and then save it, which is very inefficient with
big files, or I have to save the data in chunks using the brigade API,
which is rather inconvenient (especially for lazy Perl progammers) and
probably not as efficient as letting the OS copy the file.

I can't force the spool file to be created on the same filesystem where
I want to store my files, since TEMP_DIR is usually ignored if there's
already a temp directory specified by the environment.

The other scenario is when I want to process an uploaded file with a
random CPAN module. These don't care for the brigade API, they usually
accept file names, file handles, and maybe in-memory files. Since I
can't rely on link(), I have to pass the latter. But some modules don't
need to access the whole file, they only look at parts of it, like the
directory at the end of a Zip archive. So again this is inefficient.

When I last looked at the apreq2 source a few months ago, I was under
the impression that the code stores small uploads in memory, and only
writes them into spool files from a certain size upwards. Which might be
the reason why the filename() and fh() methods are not available
anymore. But it seemed to me that to provide link(), the code does
create files for smaller uploads when required. Couldn't this also be
done to provide filename() and fh() again?

I'd like to think that a lot of people would appreciate this, for
efficiency, simplicity and last not least backwards compatibility.

Mime
View raw message