httpd-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From PMilan...@nypl.org
Subject RE: [users@httpd] Mass Virtual Hosting
Date Fri, 20 May 2005 15:36:23 GMT



Just adding a bit more experience to the pile. Hope it is useful.
(inline)-



- how to point the ftp to the right direction, since you'll have several
machines, the data won't be centralized (would the FTP server mount all
the
disks thrugh NFS (??) or would you use AFS to have a single big
distributed
file system?? explore the possibilities)

Consider centralizing, and a deployment mechanism. Setup a large box to
centralize development, and a control mechanism (I wrote something
php/rsyncish (uses rsync currently)) that controlled what content gets
pushed to production. I also used a DB backend and some scripting on the
clients to let them know when and what to sync. I did these a group at a
time so that there is no mix of old/new code at any given point, and
redundancy is not compromised.

- what convention will you use to define where you'll store your vhosts
(on
which server, perhaps vhosts starting with A go to server A, B go to
server
B...)

This is going to be the ugly part. Are they going to be named hosts? Or
have their own configs/daemons/interfaces? You should be able to balance
them all so long as you have many many boxes. I have 5 large boxes and one
dev/staging box. I do about 1.5b hits/year. This does not include an image
farm, which gets about 1/4 of that. Vhosts are not too much of an issue in
my setup, although I do not know how it will handle your numbers. 

- how will you administrate your tons of server (to add the vhost 
definitions to
the apache conf files) perhaps take a look at cpanel, it's their business
to
make a web based application that can create the directories, apache 
conf lines
and so on when you want to add a vhost,  i know several hosting companies
use
that

Cpanel is good, but you will likely have to write some stuff to have it
deploy to all servers and do a graceful.

- Do you have the workforce to maintain your servers ?? If all your vhosts
are
quite active (you can have 1000 inactive vhosts on a single system and 
a single
vhost in one system that gets smashed under the load) you'll need an 
incredible
amount of servers

Management, management, management. Put in a hardware monitoring system,
as well as a super smart load balancing system. I built this on LVS, but I
have paperwork in for F5's Big-IP. We run dells bigger stuff on the farm,
so we use their hardware management tools which I pipe to nagios and a
couple of other tools. Another key to management is to make sure that the
servers are IDENTICAL. If I lose hardware, I can pop drives out and put
them in another chassis. In addition, bringing up another server takes me
less than an hour.

- how will you deal with logs. Centralize them. (people will want to see
how
their site is faring) and don't forget that depending on your kernel,
there is
a limitation to the amount of files aapache can open simultaneously, so
error
logs would take 1 file handler at each time. You'll need to split the
global
access.log to get one logfile per vhost, search google for the string
split-logfile)

I think this is the way to go. I have a setup (we use webtrends) that
syncs all the logs from the farm onto a windows box. I use cronolog to
split vhosts into their own hierarchies (something else cpanel may or may
not be able to do), sync them over, and they process. This happens every
half hour, so the stats are up to date. You can also archive the logs on
the log server, and through a time based gzip on them so that stats
reprocessing can be done if need be.



---------------------------------------------------------------------
The official User-To-User support forum of the Apache HTTP Server Project.
See <URL:http://httpd.apache.org/userslist.html> for more info.
To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
   "   from the digest: users-digest-unsubscribe@httpd.apache.org
For additional commands, e-mail: users-help@httpd.apache.org


---------------------------------------------------------------------
The official User-To-User support forum of the Apache HTTP Server Project.
See <URL:http://httpd.apache.org/userslist.html> for more info.
To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
   "   from the digest: users-digest-unsubscribe@httpd.apache.org
For additional commands, e-mail: users-help@httpd.apache.org


Mime
View raw message