perl-modperl mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Aleksandr Guidrevitch <pillg...@tut.by>
Subject Re: ithreads with modperl
Date Fri, 09 Jan 2004 11:50:58 GMT
Hi Simon,

Simon Clewer wrote

>Hi,
>
>We're using ithreads with modperl to run some complicated robots
>concurrently ( in a commercial environment ) - there is however an issue.
>
>Huge memory usage ... each ithread uses about 10M  of ram ( image of Apache,
>image of mod perl and image of our deep-link robot ), and as we use 5
>ithreads plus the original thread that means that each Apache is using 60 M
>and because we trade on being the best and the fastest at what we do we need
>to keep plenty of Apaches ready and waiting ( about 20 ) - so we're using
>heaps of memory.
>  
>
I can suggest  to create a frontend proxy with mod_proxy or mod_accel (a 
better aproach) -
You can keep 2-3 mod_perl processes on a backend server, and 20+ on the 
frontend,
each apache+mod_proxy process will be around 1,5Mb. With 2 Gigs on board 
you can run
20+ mod_perls and 200+ frontent server processes.

Also, i would like to suggest not to fork robots directly from apache,
but make them running as another preforked server. Your apache then
should display a refreshing page ("please wait") until a result is fetched
into cache by some of the robots.
That also means you should intensively cache robot's results where possible.

Surely, I can be completely wrong since I don't know all the details.
This is just a suggestion :)

Sincerely,
Alex



-- 
Reporting bugs: http://perl.apache.org/bugs/
Mail list info: http://perl.apache.org/maillist/modperl.html


Mime
View raw message