httpd-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ian Kluft <>
Subject [post-1.2] proposed api extension mechanism
Date Thu, 01 May 1997 09:16:25 GMT
Now that some post-1.2 ideas are showing up, I guess it's time for me to
stop lurking and toss in an idea I've been considering...  I haven't been
active on new-httpd yet but I have contributed to mod_rewrite, where I
submitted the "subrequest lookaheads".  I've also written a couple modules
for internal use on servers in Cisco's Central Engineering.

This idea could allow modules to dynamically extend the API with their own
set of functions that other code can use.  In particular, the first thing
I have in mind that could be added with this is a "lookup interface" as a
wrapper for the various DBM, DB, DBMS schemes... and then a modified mod_auth
that would use any of them (and others not yet written) via the new interface.
Other modules, such as mod_rewrite, which currently use DBM could also be
modified so they can use all available lookup methods.

How It Would Work
This would require the addition of one field to the "module" type in
http_config.h and some new support functions.  (Plus documentation updates.)

The new field in "module" would be a pointer to a hash structure, or NULL
pointer if not used by the module.  The hash would be keyed on a number or
string representing the "feature" (i.e. lookup interface, authentication
schemes, etc) and return an array of function pointers.  New "features" are
defined as a group by that array of function pointers.  Each feature would
need to specify the number of functions and exact purpose of each in its

Modules would implement features differently using their own functions in
the array.  For example, the lookup interface would define function slots
for open, lookup, close, etc which could each be done differently by modules
for DBM, DB, flat files, LDAP, various DBMS queries, etc.  Other features
would also define what their slots are used for but modules would supply
their own implementations.  So, published feature definitions would become
extensions to the Apache API.

The NULL hash pointer in the module struct is important for compatibility
with existing modules since their module structure would have a NULL filled
in for this field.

I envision these support functions.  (The "feature_id" type is up for
discussion - see implementation choice #2 below.)

* add feature support to a module: (called during module initialization to
  insert a function array in its hash table)

  int register_flex_feature( pool *p, module *mod, feature_id fid,
     array_header *func );

* call a method in every module that supports it: (similar to run_method()
  but checks hashes in each module for support of the feature)

  int run_flex_method( request_rec *r, feature_id fid, int function_index, 
     int run_all, void *param )

Obviously, in order for a feature to get used, there has to be code
(possibly in another module) that calls run_flex_method() for a specific
function from a feature definition.

The initial patch that I would provide would include
* the struct and support functions (API mods) for the Apache core
* a definition for the Lookup Interface feature's functions
* a mod_lookup_htpasswd module to implement htpasswd lookups
* a mod_lookup_dbm module to implement DBM lookups
* a mod_lookup_db module to implement Berkeley DB 1.85 lookups
* a derivative of mod_auth which uses the Lookup Interface feature

(If anyone wants to volunteer to be part of this effort, let me know.
It could go faster if we can farm out some of the pieces.)

I'll also investigate what it might take to patch mod_rewrite so that it can
use the Lookup Interface feature similar to its current dbm functionality.

That should get things started.  More lookup modules can be provided in the
future by anyone (including me.)  Hopefully this will be the last new
derivative of mod_auth and everyone else can add lookup modules. :-)

Implementation Options
I'm leaving some of these options open for discussion because there are
tradeoffs.  Also, I realize that since I'm not a member of the Apache Group,
acceptance of the proposal will require convincing members of its merits and
being open to input on these choices.

I'll watch the discussion and then put the design decisions on a web page.
>From that (and possibly more input), I'll start on the proposed patches in
the post-1.2 time frame.  (As with everyone else, this is on the side from
my day job.)

Choice #1: the hashing structure
  Pick one or suggest another alternative:
  a) This could use Apache's "table" structure if the Apache Group is
     amenable to changing "table" to some sort of hashing algorithm in
     2.0.  That would eliminate lots of sequential searches all over the
     server.  (That's a significant factor in making any software scalable!)
     If approved, I would be willing to provide this patch too.

     Possible hashing algorithms could be drawn from Berkeley DB 1.85 or 2.0
     (using in-memory hashing), or Esmond Pitt's public domain dynamic hash
     algorithm from comp.sources.misc volume 6 (with a modification to allow
     multiple instances of the structures.)  I recommend either DB 1.85 or
     2.0, but not both.  Berkeley DB 2.0 is brand new (released in April)
     but much more portable because it uses autoconf.

  b) If Option (a) is not acceptable, this algorithm still needs a hash
     function of its own in order to have variable-size and arbitrarily-keyed
     contents.  I would still suggest DB 1.85 or 2.0 if this alternative is
In case some platforms can't install the hashing library chosen, it must be
possible to use a preprocessor constant like NO_HAVE_HASHING to omit the
hash library.  In option (a), the existing sequential-search table structs
and functions would be preserved via #ifdefs.  In both cases, NO_HAVE_HASHING
would disable all flexible API extensions.  Sites defining it should still
retain all Apache 1.2 API functionality.

Choice #2: the hash key (feature ID)
  Pick one or suggest another alternative:
  a) The hash key could be an integer representing a specific feature.
     The assignment of these numbers would have to be coordinated by the
     Apache Group.  Using an integer is best for hash search performance
     so I recommend this option.
  b) The hash could also be a string.  It would not require any coordination
     except that the name strings for each feature would have to be unique.
     It could make it easier to understand these changes to the API.  But
     this is slower because of use of strcmp() for hash key matching.
Ian Kluft  KO6YQ PP-ASEL                                  Cisco Systems, Inc. (work) (home)          San Jose, CA

View raw message