Return-Path: 
 <dev-return-40371-apmail-httpd-dev-archive=httpd.apache.org@httpd.apache.org>
Delivered-To: apmail-httpd-dev-archive@www.apache.org
Received: (qmail 5192 invoked from network); 10 Dec 2003 23:54:50 -0000
Received: from daedalus.apache.org (HELO mail.apache.org) (208.185.179.12)
  by minotaur-2.apache.org with SMTP; 10 Dec 2003 23:54:50 -0000
Received: (qmail 45585 invoked by uid 500); 10 Dec 2003 23:54:30 -0000
Delivered-To: apmail-httpd-dev-archive@httpd.apache.org
Received: (qmail 45528 invoked by uid 500); 10 Dec 2003 23:54:30 -0000
Mailing-List: contact dev-help@httpd.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@httpd.apache.org
list-help: <mailto:dev-help@httpd.apache.org>
list-unsubscribe: <mailto:dev-unsubscribe@httpd.apache.org>
list-post: <mailto:dev@httpd.apache.org>
Delivered-To: mailing list dev@httpd.apache.org
Received: (qmail 45508 invoked from network); 10 Dec 2003 23:54:30 -0000
Received: from unknown (HELO mail.logilune.com) (195.154.174.52)
  by daedalus.apache.org with SMTP; 10 Dec 2003 23:54:30 -0000
Received: from stason.org (localhost.logilune.com [127.0.0.1])
	by mail.logilune.com (Postfix) with ESMTP
	id 14E147A67A; Thu, 11 Dec 2003 00:54:35 +0100 (CET)
Message-ID: <3FD7B3C6.7080800@stason.org>
Date: Wed, 10 Dec 2003 16:01:10 -0800
From: Stas Bekman <stas@stason.org>
Organization: Hope, Humanized
User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.4) Gecko/20030630
X-Accept-Language: en-us, en, he, ru
MIME-Version: 1.0
To: dev@httpd.apache.org
Cc: "Pringle, Chris (HP-PSG)" <chris.pringle@hp.com>
Subject: Re: filtering huge request bodies (like 650MB files)
References: <3FD7A3EA.7040403@stason.org> <3FD7A3EA.7040403@stason.org>
 <5.2.0.9.2.20031210171850.010c1ba8@pop3.rowe-clan.net>
In-Reply-To: <5.2.0.9.2.20031210171850.010c1ba8@pop3.rowe-clan.net>
Content-Type: text/plain; charset=us-ascii; format=flowed
Content-Transfer-Encoding: 7bit
X-Spam-Rating: daedalus.apache.org 1.6.2 0/1000/N
X-Spam-Rating: minotaur-2.apache.org 1.6.2 0/1000/N

William A. Rowe, Jr. wrote:
> At 04:57 PM 12/10/2003, Cliff Woolley wrote:
> 
>>On Wed, 10 Dec 2003, Stas Bekman wrote:
>>
>>
>>>Obviously it's not how things work at the moment, as the memory is never
>>>freed (which could probably be dealt with), but the real problem is that
>>>no data will leave the server out before it was completely read in.
>>
>>Yes, that would be the real problem.  So somewhere there is a filter (or
>>maybe the proxy itself) buffering the entire data stream before sending
>>it.  That is a bug.
> 
> 
> It's NOT the proxy - I've been through it many times - and AFAICT we have
> a simple leak in that we don't reuse the individual pool buckets, so memory
> creeps up over time.  It isn't even the end of the world, until someone at
> apachecon pointed out continous HTML proxied streams (e.g. video) really
> gobble memory, even at 8kb/min+ this isn't acceptable.
> 
> So it's not the proxy or the core output filter.  The bug lies in the Filter itself.
> Is it Chrises' own filter or one of ours?  whichever it is, it would be nice to
> get this fixed.  This is why we aught to not flip subject headers, Stas, I'm
> really too short on time to go fumbling for the original posts.  Need to know
> which filters are inserted, and therefore possibly suspect.

I wasn't flipping subjects, Bill. The original report was posted to the 
modperl list and there was indeed a leak in modperl that I've plugged 
yesterday. But Chris did testing with the fix and the leakage was still there, 
and it wasn't in his filter, since his filter didn't do anything. So the 
original post isn't relevant as it was.

Chris' filter just moves the bucket brigades through unmodified. In his case 
it was an output filter so all it did is calling ap_pass_brigade and return 
SUCCESS. His configuration was:

Listen 8081

LoadModule perl_module modules/mod_perl.so
PerlModule Apache2
PerlModule Test::BigFileTest

<VirtualHost *:8081>
	ServerName my_server_name:8081
	ServerAdmin chris.pringle@hp.com
	ProxyRequests On
	ProxyRemote * http://corporate_proxy_server:8080
	<Proxy *>
		Order Allow,Deny
		Allow from all
		PerlOutputFilterHandler Test::BigFileTest
	</Proxy>
</VirtualHost>


__________________________________________________________________
Stas Bekman            JAm_pH ------> Just Another mod_perl Hacker
http://stason.org/     mod_perl Guide ---> http://perl.apache.org
mailto:stas@stason.org http://use.perl.org http://apacheweek.com
http://modperlbook.org http://apache.org   http://ticketmaster.com