httpd-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jacob Coby" <jc...@listingbook.com>
Subject Re: [users@httpd] Need Help on Database Idea
Date Wed, 27 Nov 2002 20:34:55 GMT
>>Sorry, forgot to mention the database will be on it's own server. Probably
FreeBSD. It is meant for department heads to update there own program
information. I have done a similar project for my own web page. But I can
see a big lag because of the amount of information to be pulled for each
query. No information will be cached making it a drain on the resources.<<

I have a couple of upfront questions:

How much information is being pulled each query?
You've run these queries and seen that they take an excessive amount of
time?
Would running a dedicated (100T or GigE) network between the webserver(s)
and the database server help?

If the requirements dictate that static content *must* be generated, I would
design a system where I have a medium/low-priority background job sitting
there waiting for data to change so it can regenerate the affected page(s).

Basically, this background job would look like:

while(true)
{
  $data = GetChangedDataSince(GetLastUpdateTS());
  if(DoesPageOneDependOn($data))
    RegeneratePageOne();
}

This is how some of the major news sites (I know slashdot in particular
works this way) are able to keep up with serving huge numbers without
requiring "big iron" computers.

It also prevents some potentially hair situations when using cron jobs.  Say
you are regenerating the site every 5 min.  What happens when it takes
longer than 5 mins to regenerate the site?  Gradually you can have multiple
processes running at the same time trying to generate the same content,
slowly bringing the server to a grinding halt.

If you had the time, you could do some really slick stuff with this, and
make new pages and tables monitored with minimal effort.

Maybe with an autogen.txt to say that pageone.html should be regenerated if
either tables data_one or data_two are updated or the page is older than 2
hours. If so, pageone.html can be regenerated by calling
RegeneratePageOne().

-Jacob


---------------------------------------------------------------------
The official User-To-User support forum of the Apache HTTP Server Project.
See <URL:http://httpd.apache.org/userslist.html> for more info.
To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
   "   from the digest: users-digest-unsubscribe@httpd.apache.org
For additional commands, e-mail: users-help@httpd.apache.org


Mime
View raw message