Mailing-List: contact modperl-help@perl.apache.org; run by ezmlm
Precedence: bulk
Received-SPF: pass (herse.apache.org: domain of pharkins@gmail.com designates
 209.85.132.241 as permitted sender)
DomainKey-Signature: a=rsa-sha1; c=nofws;
        d=gmail.com; s=beta;
        h=received:message-id:date:from:sender:to:subject:cc:in-reply-to:mime-version:content-type:content-transfer-encoding:content-disposition:references:x-google-sender-auth;
        b=KxVbOBT7slx1UmL+vexWmqyCELBD1fzECWQmCQTFbNP9P8ljBSxw+0Asjx5P9Yb7ctq5sZ3sCB4C2AP1Nc66HZ8JoDK31hAzRCpTGWp9ABAUh+cuveW4O1PxXsC/9eO/IgPgvSLt8XAbUjq3dOKdAqJ7WJi/MJdhlp7v6jnLVPo=
Message-ID: <66887a3d0705062117p33d73376i8e84567e9e1c6f48@mail.gmail.com>
Date: Mon, 7 May 2007 00:17:03 -0400
From: "Perrin Harkins" <perrin@elem.com>
Sender: pharkins@gmail.com
To: "James. L" <perlwle@yahoo.com>
Subject: Re: few newbie quesitons..
Cc: modperl@perl.apache.org
In-Reply-To: <429285.85044.qm@web57911.mail.re3.yahoo.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=ISO-8859-1; format=flowed
Content-Transfer-Encoding: 7bit
Content-Disposition: inline
References: <66887a3d0705062005s66de43dbo6bc49b8580607e7f@mail.gmail.com>
	 <429285.85044.qm@web57911.mail.re3.yahoo.com>

On 5/7/07, James. L <perlwle@yahoo.com> wrote:
> the files i need to parse are usually in size of 2M -
> 10M. will the mod_perl server(2G Mem) use up the
> memory pretty quick after few hundred requests on
> different files?

You're misunderstanding a little bit.  It's not that the memory used
in parsing a file gets lost permanently.  Instead, the variable that
you loaded the data holds onto the memory from the largest size it got
to.

> sub parse {
>   my ($class,$file) = @_;
>   my @data;
>   open my $F, $file or die $!;
>   while ( my $line = <$F> ) {
>     my @fields = split /=/, $line;
>     push @data, \@fields;
>   }
>   close $F;
>   return \@data;
> }

If you read enough data into @data to use up 20MB, it will stay that
size.  That's a good thing if you intend to read another file of
similar size on the next request.  This would only be bad if you read
a very large amount of data in but only now and then.

The best way to avoid this kind of problem is to not read the whole
thing into RAM.  You can pass an iterator object to TT instead of
loading all the data at once.

- Perrin