Mailing-List: contact dev-help@apr.apache.org; run by ezmlm
Precedence: bulk
Date: Wed, 6 Jun 2001 16:28:43 +0200
From: Luke Kenneth Casson Leighton <lkcl@samba-tng.org>
To: dev@apr.apache.org
Cc: new-httpd@apache.org
Subject: XMLvL, apache, apr, mod_virgule origins: advice needed [LONG]
Message-ID: <20010606162842.V7921@angua.rince.de>
Mail-Followup-To: dev@apr.apache.org, new-httpd@apache.org
Mime-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
User-Agent: Mutt/1.2.4i

hi,

i need some help, and so i thought it best to ask here.
i am not subscribed to new-httpd@apache.org but i do
have post access, so if you reply, please cc me, but
i will keep an eye on the archives, just in case.


background
----------

raph levien developed mod_virgule (which runs advogato.org)
as a community-run forum engine.

it is 12,000 lines of hard-coded c that uses a tiny amount
of glib, a significant amount of libxml 1.8.x, and a lot
of ap_xxx routines.

raph's original comments, back in November 1999, were that
he wanted to see mod_virgule do less hard-coded c and more
scripting.

so, i started doing exactly that.  four months later,
xvl 1.0.2 was released: http://virgule.sourceforge.net.
it's still only 12,000 or so lines of code.

development stages
------------------

mod_virgule was originally an apache module.  it's actually
a web-server-in-a-web-server.  its design is therefore a
little unusual: it generates dynamic content from site xml files.

it doesn't use apache authentication, in fact there's a whole
boat-load of stuff it can't use.  it provides its own
authentication, from cookies stored in xml-based
user-profile files.

things i added:

- cgi-bin capability to run bash scripts: i wanted to
run sendmail, for example.  this required cutting
large chunks of code across from mod_cgi.

- xml-based database capabilities.  use one XML file
to specify the format of another XML file, for storage
and retrieval.  forms can be generated to do data input.

- certification-based access control.

then, i started to get it.  mod_virgule really _was_
a web-server-inside-a-web-server.  so, i added:

- socket communication (both TCP/IP and unix-domains)

and also, i started to realise, huh, this is a bit like
PHP.EXE, so why not actually split out to a separate
executable, too?

so, i:

- created a unix-domain-socket version of mod_proxy,
with an XML syntax for the communication

- used the socket communication capability to create
a socket for the ux-ds-proxy to listen to.

- split out ap_sprintf.c, alloc.c, signals.c and a
_whole_ boat-load of apache 1.3.17 source code,
making it stand-alone compilable, removing block_signals()
and all sorts of horrible stuff that i wasn't going
to need in a unix fork-only environment.

- removed the cgi-bin capability because there was 
absolutely no way i was going to cut the ap_exec()
code over, as well.

at the end of this coding spree, i had mod_virgule
as a tiny module that acts as a communication
mechanism between apache and other stand-alone
programs, which is an extremely useful thing to
have in its own right, and i had a stand-alone
executable - xvl - which was capable of running
as a mini-web-content server and a whole lot more.

things i didn't get around to doing:

- dual-compilation, ala mod_php and PHP.EXE

- re-enabling the cgi-bin capability

- working out how the hell php can rely on
so much apache code, which basically means
linking with libmain.a et al, when it's not
a public library, and how i could do the same,
instead of cutting across great swathes of
the apache codebase

- using APR and apr_util, not apache 1.3.17 code.

- using apache's HTTP code - or _whether_ to use
apache's HTTP code - in order to make xvl truly
capable of being a stand-alone web server.


problem statement
-----------------

in other words, xvl is _the_ perfect candidate
for significant integration / use of the apache
codebase, but the boundaries for doing such
radically different code generation - ala
mod_php, mod_snake, mod_xvl etc. - are not
clearly delineated or documented.

in order to progress xvl, i need to be able
to leverage the HTTP decoding and ap_xxx
codebase - without all the trappings, bells
and whistles of a fully-blown httpd
installation, if users don't want one, and
_also_ seamless integration into an existing
httpd installation, if they do.

i really want my cgi-bin capabilities back!
i want to be able to run sendmail from xvl
scripts!

i want my cake _and_ a knife, plate and spoon
_and_ the time to eat it at leisure [the cake].

please.

:)


so.  any suggestions?  what components already
exist?  i'm using a brain-dead installation
of mandrake 7.0 with glibc 2.2 (so i get
error, cannot find symbol dl_init_next@@GLIBC_2_0
whenever i compile a shared library with -ldl,
that includes libxml2, apr, pretty much damn
well everything: anyone any clues?  i'm
installing a lot of RPMs recently to overcome
this, _when_ they're available...)

xvl uses the ap_pool code, the table code,
list code, ap_psprintf, i need to exec
/ spawn programs, ideally i also need a
portable version of unix-domain-sockets, which
i understand doesn't exist in APR, yet.