httpd-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Dean Gaudet <>
Subject Re: 'Apache on {Beowolf,clusters}'
Date Tue, 08 Feb 2000 19:37:59 GMT
a CPU and some RAM on a PCI card would be about the same as an extra
CPU... or a couple extra CPUs in an SMP configuration.  except that it'd
be NUMA (non-uniform memory architecture).  you'd need some fancy
arbitration to get packets to the CPU-on-PCI... and more arbitration to
share disk i/o.  it'd be as difficult or worse to implement than SMP

SMP is frequently a waste on webservers... depends on the application
though.  if you're doing ssl, the extra cpu can really help.  if you're
just doing static stuff, the extra overhead of smp is a waste -- might as
well use a layer-3 switch and load balance between a couple disjoint
machines to crank up the i/o bandwidth.

i'm not sure what typical web apps would gain from a beowulf-style

search engines, sure.  see hotbot for a good example of a clustered search
engine.  in this case though there's a reason for the cluster architecture
-- the index of web pages is too large to hold in the memory of a single
node.  they use commodity hardware, which means 2 Gb of RAM or so per node
these days.  the index is on the order of 100Gb?  maybe 200?  i'm not sure
any more.  the nodes use myrinet to communicate with each other to share
memory and disk access.

as an exercise, consider what it'd take to write a PVM mpm for apache-2.0.  
(see i think).  the challenges will be -- what host do the
requests come into?  how do you handle failures?  how do you farm the
requests off to other hosts? pretty soon you'll start thinking about
tcp-to-tcp tunnelling.. and i don't think you'll find PVM really buys you
much at all.  at some point your architecture will start to overlap with
the "http router" concepts we've started talking about here.

there's a special case of cpu-on-a-card -- SSL accelerators.  i've been
doing some research in this area, and i'm not really sure what they gain,
but i could be wrong.  it seems to me that moore's law generally applied
to commodity cpus makes multi-cpu boxes doing SSL in software a better
choice than dedicated hardware accelerators (i.e. a dual 600Mhz x86 box
costs less and outperforms many accelerators).  but i'm biased because i
can't stand dealing with closed vendor kernel drivers, and libraries.

in our architecture at CP we do the SSL and layer-3 load balancing with
dedicated "proxy" boxes using code we wrote.  seems to work well enough.  
(we couldn't go with off-the-shelf layer-3 switches because we scale in
dimensions that other folks haven't considered yet.)


p.s. cp is hiring ;)

On Tue, 8 Feb 2000, Bill Stoddard wrote:

> > Rodent of Unusual Size wrote:
> > >
> > > I'm getting more and more requests about whether/how Apache
> > > will run on clusters.  It's becoming a FAQ.  Does anyone have
> > > any documentation on using Apache in such configurations, or
> > > on performance?  Dean? :-)
> >
> > Why would anyone want to do this?  Why not put a ton of Apache boxes
> > behind a LocalDirector or any other of the billions of load balancers.
> >
> I had the same question.  I don't know anything about Beowolf clustering,
> but if you could establish a fast comm link (at bus speeds for example)
> between the CPUs in the cluster, seems you could build some really scalable
> web server farms (with appropriate modifications to Apache to exploit the
> cluster).  Has anyone ever considered putting a CPU and some RAM on a PCI
> card, installing Linux and Apache? Seems you could scale this quite nicely.
> The PCI card Apache (with a kernel level cache?) could serve the static
> content and off-load dynamic content to a back-end server. Or visa-versa.
> The combinations are limitless. I'm suprised no one has turned this into a
> business :-)
> Bill

View raw message