httpd-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ben <...@redsnapper.net>
Subject [users@httpd] A NetHack-esque Journey of the dark arts for DBD(mysql) under httpd-2.2.3
Date Wed, 30 Aug 2006 12:53:49 GMT
This is my story of yesterday, on how I managed to bring something  
resembling mysql onto dbd over httpd 2.2.3;
It may explain why I  am astounded and amazed that 2.2.3 is called  
"The best available version of Apache", and why I believe it needs a  
lot more work to get that badge.

The story has a less than happy ending, but this epic journey of  
NetHack proportions is to be continued....

We have a production server that requires mysql based  
authentication.  We used to use one of (any number of) mysql  
authentication modules.
We have well over 100 virtual hosts on this server, using over 30  
distinct mysql database instances.

httpd 2.1 Has changed things a bit, and the Apache devteam is  
actively promoting the use of mod_dbd for all sql connections:  
Developers are told "DBD Framework (SQL Database API). [...] Apache  
2.1 and later provides the ap_dbd API for managing database  
connections, while APR 1.2 and later provides the apr_dbd API for  
interacting with the database. New modules SHOULD now use these APIs  
for all SQL database operations. Existing applications SHOULD be  
upgraded to use it where feasible, either transparently or as a  
recommended option to their users."

So - naive man that I am, it sounds like Apache's recommended  
strategy is to move over to dbd.
And - even better, Apache 2.1+ provides (to me that means provides a  
working version!) what we need, so 2.2.3 shouldn't be a problem.

What follows is my route to successfully install mysql dbd -  
overcoming typos, bugs, and so on in the process.
However, the light of day is yet to be reached.

Rather than take you minute by minute through one of the most  
convoluted days of my life, I will take you straight to the (near)  
solution. This was for some linux server - but most of the work was  
duplicated on os x. All binaries are compiled from source using gcc  
v4.0.2 (and I use tools such as wget)

FYI, all our production binaries are in the /opt directory (whereas  
by default, most stuff is installed to /usr/local) and /opt/bin is in  
$PATH.
Also on our linux, /etc/ld.so.conf includes the directories /opt/ 
lib  /opt/lib/mysql   /opt/httpd/lib /opt/httpd/apr/lib
Lastly, we also have ldap installed (which for sake of brevity, I  
won't describe the install here - but it's not a tenth as complex as  
the httpd)

Before we start, let's make sure that we have a recent version of  
autoconf (mine was out of date.. even though it was only 6 months old)
== autoconf ==
mkdir -p ~/dev
cd ~/dev
wget http://ftp.gnu.org/gnu/autoconf/autoconf-2.60.tar.gz
#why don't gnu have a 'latest' symlink for autoconf!
tar -xzf autoconf-2.60.tar.gz
cd autoconf-2.60
./configure --prefix=/opt --enable-threads=posix
make; make install

Now, we will need to make sure that our mysql server has the thread- 
safe client library. Mine didn't.
We already had a 5 server so I only needed a reboot.
============= mysql =============
cd ~/dev
wget http://dev.mysql.com/get/Downloads/MySQL-5.0/mysql-5.0.24.tar.gz/ 
from/ftp://ftp.mirrorservice.org/sites/ftp.mysql.com/
tar -xzf mysql-5.0.24.tar.gz
cd mysql-5.0.24
CFLAGS="-O3" CXX=gcc CXXFLAGS="-O3 -felide-constructors -fno- 
exceptions -fno-rtti" ./configure --prefix=/opt --with-charset=utf8 -- 
enable-thread-safe-client --enable-threads=posix --with-ncurses
make all; make install
sudo ldconfig -v   #linux only
mysqladmin shutdown;  mysqld_safe&

Now to install httpd (at last)
============= httpd =============
cd ~/dev
wget http://apache.rmplc.co.uk/httpd/httpd-2.2.3.tar.gz
tar -xzf httpd-2.2.3.tar.gz
#Apache devteam. don't like us using nph- anymore. But I live in the  
real world and have legacy systems to look after.
cd httpd-2.2.3/modules/generators
wget  -O mod_cgi.patch "http://issues.apache.org/bugzilla/ 
attachment.cgi?id=17891"
cp mod_cgi.c mod_cgi.c.orig
patch mod_cgi.c mod_cgi.patch
#Now I need to manually change the install locations for apr and apr- 
util
cd ~/dev/httpd-2.2.3/srclib/apr
vi config.layout  # (line 14 - in Layout apr, change prefix to /opt/ 
httpd/apr )
cd ~/dev/httpd-2.2.3/srclib/apr-util
vi config.layout  # (line 14 - in Layout apr, change prefix to /opt/ 
httpd/apr )
#Now I need to get the mysql driver
cd ~/dev/httpd-2.2.3/srclib/apr-util/dbd
wget http://apache.webthing.com/svn/apache/apr/apr_dbd_mysql.c
vi  apr_dbd_mysql.c  #Remove mysql header conditional lines  
48,49,50,51,54 - keep #include <mysql/mysql.h> and #include <mysql/ 
errmsg.h>
cd ~/dev/httpd-2.2.3/srclib/apr-util
./buildconf       #Otherwise the mysql file will not be found.
#we can test to see if the mysql driver will be loaded now by doing
# make; ldd .libs/libaprutil-1.so | grep libmysqlclient_r
cd ~/dev/httpd-2.2.3/srclib/apr-util/build
#Now we need to patch the bugged dbd.m4 file  -- we can spend hours  
reading the threads regarding this issue - if we can find them.
wget -O dbd.patch -p0 "http://marc2.theaimsgroup.com/?l=apr- 
dev&m=114384830419072&q=p3"
cp dbd.m4 dbd.m4.orig
patch -p0 dbd.m4 dbd.patch  #(Use ctrl-d to skip the second patch.)
#Now we need to move any current directory (to stop some silly logic  
from ruining the day - nothing to do with symlinks )
mv /opt/httpd /opt/httpd.x
cd ~/dev/httpd-2.2.3
./buildconf
vi config.layout #( Copy the 22 lines of the Apache Layout and call  
it  Opt - change just the prefix line to  opt/httpd )
./configure --prefix=/opt/httpd --enable-layout=Opt --enable-dbd -- 
with-mysql=/opt  --enable-access --enable-ldap  --enable-actions -- 
enable-alias --enable-auth --enable-auth_dbm --enable-auth_digest -- 
enable-authn_dbd --enable-authnz-ldap --enable-cache --enable-cgi -- 
enable-dir --enable-disk_cache --enable-dumpio --enable-env --enable- 
expires --enable-fastcgi --enable-file_cache --enable-headers -- 
enable-include --enable-info --enable-log_config --enable- 
log_forensic --enable-logio --enable-mem_cache --enable-mime --enable- 
mime_magic --enable-negotiation  --enable-rewrite --enable-setenvif -- 
enable-ssl --enable-status --enable-unique_id --enable-usertrack -- 
enable-ldap  --enable-version --enable-vhost_alias --enable-so -- 
enable-module=all --enable-shared=max
make
#Now move back the directory (To keep conf files)
mv /opt/httpd.x /opt/httpd
make install
--------------------------------------------------
Phew.  Now to set up the conf files.

httpd.conf

DBDriver mysql
# DBDPersist Off     #Off/On Not 0 or 1  !!! When we RTFM, we must  
realise it is way out of date. !!!
DBDMin  1
DBDKeep 2             #This isn't generating DBD for each VirtualHost  
by any chance?
DBDMax  20
DBDExptime 10

#If you stick the above values into a VirtualHost directive, httpd  
will die catastrophically.  The sign of it dying is the following log  
messages:
#Believe me, the following messages occur if the values are put in  
the context of a VirtualHost.  The driver is definitely installed and  
loaded.
  [crit] (70023)This function has not been implemented on this  
platform: DBD: driver for [DBDriver unset] not available
  [crit] (70023)This function has not been implemented on this  
platform: DBD: failed to initialise

#Now we need to stick the DBDParams under each VirtualHost
DBDParams  "dbname=client005 user=dbdauthent"  #no password etc needed.
#And the following (straight out from RTFM )
<Directory />
  AuthType Basic
  AuthName "Database access"
  AuthBasicProvider dbd
  Require valid-user
  AuthDBDUserPWQuery "select password from authn where username = %s"
</Directory>

----------------------------------------
Restart the server..
Whee! And I get a prompt for my username and password..
but..  but..  but.. Ho!!  Put in my user details and what's this? A  
500 (Internal Server Error)? Surely with LogLevel debug, I have  
something about this?
[error] Error looking up testuser in database
Well, that's informative. So much for LogLevel debug
What else happens - that shouldn't?
Everytime I restart the server, I am swamped with additional httpd  
threads - which kill the server. Coming from DBD somewhere (when I  
comment out dbd, the problem goes) - are DBDMin/ DVDKeep installing  
new threads for each virtualhost? 'cos something seriously weird is  
going on...

----------------------------------------
Current conclusion.  The apache devteam are forgetting to implement  
QA in their releases.
They should not be recommending a set of tools that are hard to  
implement and subsequently fail.
The documentation for dbd is is out of date, sparse, uninformative,  
and distributed all over the place.
(At least one Apache developer I know is amazed if one hasn't read  
every single text file in every directory of the release! Have you  
looked through every directory of the release)
Moreover - much of the documentation assumes that we know which  
directory things are to go into.

This epic journey is to be continued....



---------------------------------------------------------------------
The official User-To-User support forum of the Apache HTTP Server Project.
See <URL:http://httpd.apache.org/userslist.html> for more info.
To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
   "   from the digest: users-digest-unsubscribe@httpd.apache.org
For additional commands, e-mail: users-help@httpd.apache.org


Mime
View raw message