perl-modperl mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Randolf Richardson" <rand...@modperl.pl>
Subject Re: Large File Download
Date Sat, 28 Mar 2015 16:45:34 GMT
> Randolf Richardson wrote:
> >> I know that it's possible(and arguably best practice) to use Apache to
> >> download large files efficiently and quickly, without passing them through
> >> mod_perl. However, the data I need to download from my application is both
> >> dynamically generated and sensitive so I cannot expose it to the internet
> >> for anonymous download via Apache. So, I'm wondering if mod_perl has a
> >> capability similar to the output stream of a java servlet. Specifically, I
> >> want to return bits and pieces of the file at a time over the wire so that
> >> I can avoid loading the entire file into memory prior to sending it to the
> >> browser. Currently, I'm loading the entire file into memory before sending
> >> it and
> >>
> >> Is this possible with mod_perl and, if so, how should I go about
> >> implementing it?
> > 
> > 	Yes, it is possible -- instead of loading the entire contents of a 
> > file into RAM, just read blocks in a loop and keep sending them until 
> > you reach EoF (End of File).
> > 
> > 	You can also use $r->flush along the way if you like, but as I 
> > understand it this isn't necessary because Apache HTTPd will send the 
> > data as soon as its internal buffers contain enough data.  Of course, 
> > if you can tune your block size in your loop to match Apache's output 
> > buffer size, then that will probably help.  (I don't know much about 
> > the details of Apache's output buffers because I've not read up too 
> > much on them, so I hope my assumptions about this are correct.)
> > 
> > 	One of the added benefits you get from using a loop is that you can 
> > also implement rate limiting if that becomes useful.  You can 
> > certainly also implement access controls as well by cross-checking 
> > the file being sent with whatever internal database queries you'd 
> > normally use to ensure it's okay to send the file first.
> > 
> 
> You can also :
> 1) write the data to a file
> 2) $r->sendfile(...);
> 3) add a cleanup handler, to delete the file when the request has been served.
> See here for details : http://perl.apache.org/docs/2.0/api/Apache2/RequestIO.html#C_sendfile_
> 
> For this to work, there is an Apache configuration directive which must be set to "on".
I 
> believe it is called "UseSendFile".
> Essentially what senfile() does, is to delegate the actual reading and sending of the
file 
> to Apache httpd and the underlying OS, using code which is specifically optimised for
this 
> purpose.  It is much kore efficient than doing this in a read/write loop by yourself,
at 
> the cost of having less fine control over the operation.

	Thank you André, this is an excellent solution.

	One comment I'd like to add is that it appears that steps 1 and 3 
can be eliminated if a file is basically being copied (because the 
data is pre-existing in its entirety in a file already), hence 
resulting in even better performance due to even fewer resources and 
CPU cycles being consumed.

Randolf Richardson - randolf@inter-corporate.com
Inter-Corporate Computer & Network Services, Inc.
Beautiful British Columbia, Canada
http://www.inter-corporate.com/



Mime
View raw message