perl-modperl mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jonathan Vanasco <modperl-l...@2xlp.com>
Subject Re: Loading at startup to use shared memory
Date Tue, 07 Nov 2006 16:35:52 GMT

On Nov 7, 2006, at 6:57 AM, Kjetil Kjernsmo wrote:
> Exactly what numbers are you then reading out of top then?

Run your app on a clean box -- ie: don't run anything other than  
apache/mp ( if you can toss the db onto another service , do it ).   
but turn off everything else.

Reboot your machine into that state, so all the memory is  
'free' ( nothing is marked as used when its free, and then recycled )

then watch the 'free' system memory as you start / stop apache.  if  
all goes well, you'll have no memory 'leakage' - it'll stop to the  
same amount of free ram as before.  do a few test runs ( its not  
really a leak, the memory is free , but the system sometimes doesn't  
show it as free ).  profile your code, if needed, to get things to  
show like that.

then just calculate differences on each round of server configurations.

also:

using the various apache sizing tools like Apache2::Status with all  
the bells and whistles, you can profile where code /memory is wasted

some tips:

	avoid cpan modules not intended for persistent  frameworks if you can.
		when you use cpan stuff, opt for ones that wrap c functions.
		just weigh the benefit from the drawbacks.
			things like DBI , Rose/DBIx, Date::Calc, etc are obviously integral.
			but things like File::Find eat up a TON of memory ( 2+ mb ), and  
most functionality can be coded in 10-15 minutes.  ( vs 2 minutes if  
you use the module )

	use all of your internal code with constants for debugging, not  
variables
		if ( DEBUG__ ) { }	is optimized away
		if ( $DEBUG__ ) {} is not
		if you want to use variables, wrap them in a constant
			if ( DEBUG ) {
				if ( $DEBUG__ ) {}
			}
		your dev machine takes a hit on performance with this approach, but  
your production machine is way leaner ( 30-40% less opcodes and  
memory ) than the standard $DEBUG methods

	use refs whenever you can.
		i have a custom page renderer.  it doesn't print directly to $r,  
because it builds page components ( usually out of Petal but  
sometimes other stuff too ) then aggregates them together.
		switching to a ref based system saved me ~3mb of memory per child  
on the render section alone.  that was about 5 lines of code to  
implement.

	pay close attention to what is loaded pre-fork and unshared post-fork
		if you hit a  certain page, and your unshared memory in that child  
jumps up 10+mb, you should look into exactly what is causing that  
jump.  you probably have something that can be optimized

	Find your 'baseline' apache memory imprints
		Run single server mode.  Note the size of memory before any page  
hits.  Then note growth as you hit the system.
		Run multi server mode.  Note the size of the parent process, the  
child process BEFORE any hits.  Then note the size of both after a hit.
		
		In my system,
			I'm up to ~110 parent memory.
			Immediately after the fork, each child is ~5mb.
			Immediately after the first request, each child is ~12mb.
			subsequent requests grow a few k / mb depending on output.  i kill  
them @500 requests per child or 20MB, whichever comes first (under  
normal use, its the same )
	
	Note that DBI and some other modules do caching.
		DBI caches db handles and statement handles internally.  i think  
they reset @ 120.
		usually that means you see 1) process growth from old handles  
sticking around ; 2) leaked SVs
		There's nothing you can /need to do about it, aside from not worry.





// Jonathan Vanasco

| - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -  
- - - - - - - - - - - - - - - -
| FindMeOn.com - The cure for Multiple Web Personality Disorder
| Web Identity Management and 3D Social Networking
| - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -  
- - - - - - - - - - - - - - - -
| RoadSound.com - Tools For Bands, Stuff For Fans
| Collaborative Online Management And Syndication Tools
| - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -  
- - - - - - - - - - - - - - - -



Mime
View raw message