Return-Path: Delivered-To: apmail-httpd-dev-archive@www.apache.org Received: (qmail 87985 invoked from network); 12 Sep 2003 12:02:08 -0000 Received: from daedalus.apache.org (HELO mail.apache.org) (208.185.179.12) by minotaur-2.apache.org with SMTP; 12 Sep 2003 12:02:08 -0000 Received: (qmail 55867 invoked by uid 500); 12 Sep 2003 12:01:54 -0000 Delivered-To: apmail-httpd-dev-archive@httpd.apache.org Received: (qmail 55819 invoked by uid 500); 12 Sep 2003 12:01:53 -0000 Mailing-List: contact dev-help@httpd.apache.org; run by ezmlm Precedence: bulk Reply-To: dev@httpd.apache.org list-help: list-unsubscribe: list-post: Delivered-To: mailing list dev@httpd.apache.org Received: (qmail 55782 invoked from network); 12 Sep 2003 12:01:50 -0000 Received: from unknown (HELO localhost.localdomain) (62.212.108.2) by daedalus.apache.org with SMTP; 12 Sep 2003 12:01:50 -0000 Received: from [192.168.100.1] (auxiliance-2.clients.easynet.fr [212.180.95.194]) by localhost.localdomain (8.12.8/8.12.8) with ESMTP id h8CBgAgn025271 for ; Fri, 12 Sep 2003 13:42:12 +0200 Subject: [PATCH] Bug #18756: fix many problems about ldap cache using shared memory From: Matthieu Estrade To: dev@httpd.apache.org Content-Type: multipart/mixed; boundary="=-hTwz1xuPwY44dTaCk1Kj" Organization: Message-Id: <1063368100.2277.25.camel@matthieu-dev> Mime-Version: 1.0 X-Mailer: Ximian Evolution 1.2.2 (1.2.2-5) Date: 12 Sep 2003 14:01:41 +0200 X-MailScanner-Information: Please contact the ISP for more information X-MailScanner: Found to be clean X-Spam-Rating: daedalus.apache.org 1.6.2 0/1000/N X-Spam-Rating: minotaur-2.apache.org 1.6.2 0/1000/N --=-hTwz1xuPwY44dTaCk1Kj Content-Type: text/plain Content-Transfer-Encoding: 7bit Hi, This is my second and new patch for ldap cache using shared memory. I am using ldap cache with linux redhat and worker mpm. The problems i found: 1) All shm cache information (shm addr, rmm addr, lock) are declared as global. --> in worker mpm, many strange problems happen. 2) shm initialisation is in child init hook --> it try to create an shm file per child 3) ldap cache using shm is using a default filename to create shm, in /tmp --> if 2 different httpd are used, only one can use ldap cache using shm. What i did: 1) i moved all this global declaration inside the module config, then i made all rmm_alloc and rmm_free use these module config data. 2) i moved the shm init in post_config hook, then did some code to merge all these rmm and shm addr in all vhost module config 3) i created a directive LDAPSharedCacheFile receiving a filename, to create the file with apr_root_relative which will be used to create the shm. I tested this patch on linux rh9 with 3 vhost using ldap auth, it work well. I also tested the ldap-status and it work well too. Feedback are welcome ! regards, Matthieu --=-hTwz1xuPwY44dTaCk1Kj Content-Disposition: attachment; filename=patch_ldap_cache.diff Content-Type: text/plain; name=patch_ldap_cache.diff; charset=UTF-8 Content-Transfer-Encoding: 7bit ? patch_ldap_cache.diff Index: include/util_ldap.h =================================================================== RCS file: /home/cvspublic/httpd-2.0/include/util_ldap.h,v retrieving revision 1.11 diff -u -r1.11 util_ldap.h --- include/util_ldap.h 14 Feb 2003 16:04:00 -0000 1.11 +++ include/util_ldap.h 12 Sep 2003 10:08:11 -0000 @@ -75,6 +75,10 @@ #include "http_protocol.h" #include "http_request.h" +#if APR_HAS_SHARED_MEMORY +#include "apr_rmm.h" +#include "apr_shm.h" +#endif /* Create a set of LDAP_DECLARE(type), LDLDAP_DECLARE(type) and * LDAP_DECLARE_DATA with appropriate export and import tags for the platform @@ -97,7 +101,6 @@ #define LDAP_DECLARE_DATA __declspec(dllimport) #endif - /* * LDAP Connections */ @@ -138,9 +141,11 @@ apr_pool_t *pool; /* pool from which this state is allocated */ #if APR_HAS_THREADS apr_thread_mutex_t *mutex; /* mutex lock for the connection list */ + apr_thread_rwlock_t *util_ldap_cache_lock; #endif apr_size_t cache_bytes; /* Size (in bytes) of shared memory cache */ + char *cache_file; /* filename for shm */ long search_cache_ttl; /* TTL for search cache */ long search_cache_size; /* Size (in entries) of search cache */ long compare_cache_ttl; /* TTL for compare cache */ @@ -150,6 +155,15 @@ char *cert_auth_file; int cert_file_type; int ssl_support; + +#if APR_HAS_SHARED_MEMORY + apr_shm_t *cache_shm; + apr_rmm_t *cache_rmm; +#endif + + /* cache ald */ + void *util_ldap_cache; + } util_ldap_state_t; @@ -286,21 +300,21 @@ * @param reqsize The size of the shared memory segement to request. A size * of zero requests the max size possible from * apr_shmem_init() - * @deffunc void util_ldap_cache_init(apr_pool_t *p) + * @deffunc void util_ldap_cache_init(apr_pool_t *p, util_ldap_state_t *st) * @return The status code returned is the status code of the * apr_smmem_init() call. Regardless of the status, the cache * will be set up at least for in-process or in-thread operation. */ -apr_status_t util_ldap_cache_init(apr_pool_t *pool, apr_size_t reqsize); +apr_status_t util_ldap_cache_init(apr_pool_t *pool, util_ldap_state_t *st); /** * Display formatted stats for cache * @param The pool to allocate the returned string from * @tip This function returns a string allocated from the provided pool that describes * various stats about the cache. - * @deffunc char *util_ald_cache_display(apr_pool_t *pool) + * @deffunc char *util_ald_cache_display(apr_pool_t *pool, util_ldap_state_t *st) */ -char *util_ald_cache_display(apr_pool_t *pool); +char *util_ald_cache_display(apr_pool_t *pool, util_ldap_state_t *st); /* from apr_ldap_cache_mgr.c */ @@ -310,9 +324,9 @@ * @param The pool to allocate the returned string from * @tip This function returns a string allocated from the provided pool that describes * various stats about the cache. - * @deffunc char *util_ald_cache_display(apr_pool_t *pool) + * @deffunc char *util_ald_cache_display(apr_pool_t *pool, util_ldap_state_t *st) */ -char *util_ald_cache_display(apr_pool_t *pool); +char *util_ald_cache_display(apr_pool_t *pool, util_ldap_state_t *st); #endif /* APU_HAS_LDAP */ #endif /* UTIL_LDAP_H */ Index: modules/experimental/util_ldap.c =================================================================== RCS file: /home/cvspublic/httpd-2.0/modules/experimental/util_ldap.c,v retrieving revision 1.14 diff -u -r1.14 util_ldap.c --- modules/experimental/util_ldap.c 4 Apr 2003 13:47:13 -0000 1.14 +++ modules/experimental/util_ldap.c 12 Sep 2003 10:08:16 -0000 @@ -141,6 +141,7 @@ */ int util_ldap_handler(request_rec *r) { + util_ldap_state_t *st = (util_ldap_state_t *)ap_get_module_config(r->server->module_config, &ldap_module); r->allowed |= (1 << M_GET); if (r->method_number != M_GET) @@ -171,7 +172,7 @@ "\n", r ); - ap_rputs(util_ald_cache_display(r->pool), r); + ap_rputs(util_ald_cache_display(r->pool, st), r); ap_rputs("\n

\n", r); @@ -506,9 +507,7 @@ LDAPMessage *res, *entry; char *searchdn; - util_ldap_state_t *st = - (util_ldap_state_t *)ap_get_module_config(r->server->module_config, - &ldap_module); + util_ldap_state_t *st = (util_ldap_state_t *)ap_get_module_config(r->server->module_config, &ldap_module); /* read lock this function */ LDAP_CACHE_LOCK_CREATE(st->pool); @@ -517,7 +516,7 @@ LDAP_CACHE_WRLOCK(); curnode.url = url; - curl = util_ald_cache_fetch(util_ldap_cache, &curnode); + curl = util_ald_cache_fetch(st->util_ldap_cache, &curnode); if (curl == NULL) { curl = util_ald_create_caches(st, url); } @@ -637,7 +636,7 @@ /* get cache entry (or create one) */ LDAP_CACHE_WRLOCK(); curnode.url = url; - curl = util_ald_cache_fetch(util_ldap_cache, &curnode); + curl = util_ald_cache_fetch(st->util_ldap_cache, &curnode); if (curl == NULL) { curl = util_ald_create_caches(st, url); } @@ -760,7 +759,7 @@ /* Get the cache node for this url */ LDAP_CACHE_WRLOCK(); curnode.url = url; - curl = (util_url_node_t *)util_ald_cache_fetch(util_ldap_cache, &curnode); + curl = (util_url_node_t *)util_ald_cache_fetch(st->util_ldap_cache, &curnode); if (curl == NULL) { curl = util_ald_create_caches(st, url); } @@ -962,6 +961,24 @@ return NULL; } +static const char *util_ldap_set_cache_file(cmd_parms *cmd, void *dummy, const char *file) +{ + util_ldap_state_t *st = + (util_ldap_state_t *)ap_get_module_config(cmd->server->module_config, + &ldap_module); + + if (file) + st->cache_file = ap_server_root_relative(st->pool, file); + else + st->cache_file = NULL; + + ap_log_error(APLOG_MARK, APLOG_DEBUG|APLOG_NOERRNO, 0, cmd->server, + "LDAP cache: Setting shared memory cache file to %s bytes.", + st->cache_file); + + return NULL; +} + static const char *util_ldap_set_cache_ttl(cmd_parms *cmd, void *dummy, const char *ttl) { util_ldap_state_t *st = @@ -1091,22 +1108,6 @@ return st; } -static void util_ldap_init_module(apr_pool_t *pool, server_rec *s) -{ - util_ldap_state_t *st = - (util_ldap_state_t *)ap_get_module_config(s->module_config, - &ldap_module); - - apr_status_t result = util_ldap_cache_init(pool, st->cache_bytes); - char buf[MAX_STRING_LEN]; - - apr_strerror(result, buf, sizeof(buf)); - ap_log_error(APLOG_MARK, APLOG_DEBUG|APLOG_NOERRNO, result, s, - "[%d] ldap cache init: %s", - getpid(), buf); -} - - static apr_status_t util_ldap_cleanup_module(void *data) { server_rec *s = data; @@ -1128,11 +1129,39 @@ apr_pool_t *ptemp, server_rec *s) { int rc = LDAP_SUCCESS; + apr_status_t result; + char buf[MAX_STRING_LEN]; - util_ldap_state_t *st = (util_ldap_state_t *)ap_get_module_config( - s->module_config, - &ldap_module); + server_rec *s_vhost; + util_ldap_state_t *st_vhost; + + util_ldap_state_t *st = (util_ldap_state_t *)ap_get_module_config(s->module_config, &ldap_module); + /* initializing cache if file is here and we already don't have shm addr*/ + if (st->cache_file && !st->cache_shm) { + result = util_ldap_cache_init(p, st); + apr_strerror(result, buf, sizeof(buf)); + ap_log_error(APLOG_MARK, APLOG_DEBUG|APLOG_NOERRNO, result, s, "LDAP cache init: %s",buf); + + + /* merge config in all vhost */ + s_vhost = s->next; + while (s_vhost) { + ap_log_error(APLOG_MARK, APLOG_DEBUG|APLOG_NOERRNO, result, s, + "LDAP merging Shared Cache conf: shm=0x%x rmm=0x%x for VHOST: %s", + st->cache_shm, st->cache_rmm, s_vhost->server_hostname); + + st_vhost = (util_ldap_state_t *)ap_get_module_config(s_vhost->module_config, &ldap_module); + st_vhost->cache_shm = st->cache_shm; + st_vhost->cache_rmm = st->cache_rmm; + st_vhost->cache_file = st->cache_file; + s_vhost = s_vhost->next; + } + + } else { + ap_log_error(APLOG_MARK, APLOG_DEBUG|APLOG_NOERRNO, 0 , s, "LDAP cache: Unable to init Shared Cache: no file"); + } + /* log the LDAP SDK used */ #if APR_HAS_NETSCAPE_LDAPSDK @@ -1297,6 +1326,10 @@ "Sets the size of the shared memory cache in bytes. " "Zero means disable the shared memory cache. Defaults to 100KB."), + AP_INIT_TAKE1("LDAPSharedCacheFile", util_ldap_set_cache_file, NULL, RSRC_CONF, + "Sets the file of the shared memory cache." + "Nothing means disable the shared memory cache."), + AP_INIT_TAKE1("LDAPCacheEntries", util_ldap_set_cache_entries, NULL, RSRC_CONF, "Sets the maximum number of entries that are possible in the LDAP " "search cache. " @@ -1331,7 +1364,6 @@ static void util_ldap_register_hooks(apr_pool_t *p) { ap_hook_post_config(util_ldap_post_config,NULL,NULL,APR_HOOK_MIDDLE); - ap_hook_child_init(util_ldap_init_module, NULL, NULL, APR_HOOK_MIDDLE); ap_hook_handler(util_ldap_handler, NULL, NULL, APR_HOOK_MIDDLE); } Index: modules/experimental/util_ldap_cache.c =================================================================== RCS file: /home/cvspublic/httpd-2.0/modules/experimental/util_ldap_cache.c,v retrieving revision 1.8 diff -u -r1.8 util_ldap_cache.c --- modules/experimental/util_ldap_cache.c 21 Jan 2003 10:53:09 -0000 1.8 +++ modules/experimental/util_ldap_cache.c 12 Sep 2003 10:08:17 -0000 @@ -85,14 +85,14 @@ return(strcmp(na->url, nb->url) == 0); } -void *util_ldap_url_node_copy(void *c) +void *util_ldap_url_node_copy(util_ald_cache_t *cache, void *c) { util_url_node_t *n = (util_url_node_t *)c; - util_url_node_t *node = (util_url_node_t *)util_ald_alloc(sizeof(util_url_node_t)); + util_url_node_t *node = (util_url_node_t *)util_ald_alloc(cache->rmm_addr, sizeof(util_url_node_t)); if (node) { - if (!(node->url = util_ald_strdup(n->url))) { - util_ald_free(node->url); + if (!(node->url = util_ald_strdup(cache->rmm_addr, n->url))) { + util_ald_free(cache->rmm_addr, node->url); return NULL; } node->search_cache = n->search_cache; @@ -105,15 +105,15 @@ } } -void util_ldap_url_node_free(void *n) +void util_ldap_url_node_free(util_ald_cache_t *cache, void *n) { util_url_node_t *node = (util_url_node_t *)n; - util_ald_free(node->url); + util_ald_free(cache->rmm_addr, node->url); util_ald_destroy_cache(node->search_cache); util_ald_destroy_cache(node->compare_cache); util_ald_destroy_cache(node->dn_compare_cache); - util_ald_free(node); + util_ald_free(cache->rmm_addr, node); } /* ------------------------------------------------------------------ */ @@ -131,10 +131,10 @@ ((util_search_node_t *)b)->username) == 0); } -void *util_ldap_search_node_copy(void *c) +void *util_ldap_search_node_copy(util_ald_cache_t *cache, void *c) { util_search_node_t *node = (util_search_node_t *)c; - util_search_node_t *newnode = util_ald_alloc(sizeof(util_search_node_t)); + util_search_node_t *newnode = util_ald_alloc(cache->rmm_addr, sizeof(util_search_node_t)); /* safety check */ if (newnode) { @@ -144,13 +144,13 @@ int k = 0; int i = 0; while (node->vals[k++]); - if (!(newnode->vals = util_ald_alloc(sizeof(char *) * (k+1)))) { - util_ldap_search_node_free(newnode); + if (!(newnode->vals = util_ald_alloc(cache->rmm_addr, sizeof(char *) * (k+1)))) { + util_ldap_search_node_free(cache, newnode); return NULL; } while (node->vals[i]) { - if (!(newnode->vals[i] = util_ald_strdup(node->vals[i]))) { - util_ldap_search_node_free(newnode); + if (!(newnode->vals[i] = util_ald_strdup(cache->rmm_addr, node->vals[i]))) { + util_ldap_search_node_free(cache, newnode); return NULL; } i++; @@ -159,10 +159,10 @@ else { newnode->vals = NULL; } - if (!(newnode->username = util_ald_strdup(node->username)) || - !(newnode->dn = util_ald_strdup(node->dn)) || - !(newnode->bindpw = util_ald_strdup(node->bindpw)) ) { - util_ldap_search_node_free(newnode); + if (!(newnode->username = util_ald_strdup(cache->rmm_addr, node->username)) || + !(newnode->dn = util_ald_strdup(cache->rmm_addr, node->dn)) || + !(newnode->bindpw = util_ald_strdup(cache->rmm_addr, node->bindpw)) ) { + util_ldap_search_node_free(cache, newnode); return NULL; } newnode->lastbind = node->lastbind; @@ -171,20 +171,20 @@ return (void *)newnode; } -void util_ldap_search_node_free(void *n) +void util_ldap_search_node_free(util_ald_cache_t *cache, void *n) { int i = 0; util_search_node_t *node = (util_search_node_t *)n; if (node->vals) { while (node->vals[i]) { - util_ald_free(node->vals[i++]); + util_ald_free(cache->rmm_addr, node->vals[i++]); } - util_ald_free(node->vals); + util_ald_free(cache->rmm_addr, node->vals); } - util_ald_free(node->username); - util_ald_free(node->dn); - util_ald_free(node->bindpw); - util_ald_free(node); + util_ald_free(cache->rmm_addr, node->username); + util_ald_free(cache->rmm_addr, node->dn); + util_ald_free(cache->rmm_addr, node->bindpw); + util_ald_free(cache->rmm_addr, node); } /* ------------------------------------------------------------------ */ @@ -204,16 +204,16 @@ strcmp(na->value, nb->value) == 0); } -void *util_ldap_compare_node_copy(void *c) +void *util_ldap_compare_node_copy(util_ald_cache_t *cache, void *c) { util_compare_node_t *n = (util_compare_node_t *)c; - util_compare_node_t *node = (util_compare_node_t *)util_ald_alloc(sizeof(util_compare_node_t)); + util_compare_node_t *node = (util_compare_node_t *)util_ald_alloc(cache->rmm_addr, sizeof(util_compare_node_t)); if (node) { - if (!(node->dn = util_ald_strdup(n->dn)) || - !(node->attrib = util_ald_strdup(n->attrib)) || - !(node->value = util_ald_strdup(n->value))) { - util_ldap_compare_node_free(node); + if (!(node->dn = util_ald_strdup(cache->rmm_addr, n->dn)) || + !(node->attrib = util_ald_strdup(cache->rmm_addr, n->attrib)) || + !(node->value = util_ald_strdup(cache->rmm_addr, n->value))) { + util_ldap_compare_node_free(cache, node); return NULL; } node->lastcompare = n->lastcompare; @@ -225,13 +225,13 @@ } } -void util_ldap_compare_node_free(void *n) +void util_ldap_compare_node_free(util_ald_cache_t *cache, void *n) { util_compare_node_t *node = (util_compare_node_t *)n; - util_ald_free(node->dn); - util_ald_free(node->attrib); - util_ald_free(node->value); - util_ald_free(node); + util_ald_free(cache->rmm_addr, node->dn); + util_ald_free(cache->rmm_addr, node->attrib); + util_ald_free(cache->rmm_addr, node->value); + util_ald_free(cache->rmm_addr, node); } /* ------------------------------------------------------------------ */ @@ -247,14 +247,14 @@ ((util_dn_compare_node_t *)b)->reqdn) == 0); } -void *util_ldap_dn_compare_node_copy(void *c) +void *util_ldap_dn_compare_node_copy(util_ald_cache_t *cache, void *c) { util_dn_compare_node_t *n = (util_dn_compare_node_t *)c; - util_dn_compare_node_t *node = (util_dn_compare_node_t *)util_ald_alloc(sizeof(util_dn_compare_node_t)); + util_dn_compare_node_t *node = (util_dn_compare_node_t *)util_ald_alloc(cache->rmm_addr, sizeof(util_dn_compare_node_t)); if (node) { - if (!(node->reqdn = util_ald_strdup(n->reqdn)) || - !(node->dn = util_ald_strdup(n->dn))) { - util_ldap_dn_compare_node_free(node); + if (!(node->reqdn = util_ald_strdup(cache->rmm_addr, n->reqdn)) || + !(node->dn = util_ald_strdup(cache->rmm_addr, n->dn))) { + util_ldap_dn_compare_node_free(cache, node); return NULL; } return node; @@ -264,12 +264,12 @@ } } -void util_ldap_dn_compare_node_free(void *n) +void util_ldap_dn_compare_node_free(util_ald_cache_t *cache, void *n) { util_dn_compare_node_t *node = (util_dn_compare_node_t *)n; - util_ald_free(node->reqdn); - util_ald_free(node->dn); - util_ald_free(node); + util_ald_free(cache->rmm_addr, node->reqdn); + util_ald_free(cache->rmm_addr, node->dn); + util_ald_free(cache->rmm_addr, node); } @@ -279,42 +279,48 @@ apr_status_t util_ldap_cache_module_kill(void *data) { + util_ldap_state_t *st = (util_ldap_state_t *)data; + #if APR_HAS_SHARED_MEMORY - if (util_ldap_shm != NULL) { - apr_status_t result = apr_shm_destroy(util_ldap_shm); - util_ldap_shm = NULL; + if (st->cache_shm != NULL) { + apr_status_t result = apr_shm_destroy(st->cache_shm); + st->cache_shm = NULL; + unlink(st->cache_file); return result; } #endif - util_ald_destroy_cache(util_ldap_cache); + util_ald_destroy_cache(st->util_ldap_cache); return APR_SUCCESS; } -apr_status_t util_ldap_cache_init(apr_pool_t *pool, apr_size_t reqsize) +apr_status_t util_ldap_cache_init(apr_pool_t *pool, util_ldap_state_t *st) { #if APR_HAS_SHARED_MEMORY apr_status_t result; - result = apr_shm_create(&util_ldap_shm, reqsize, MODLDAP_SHMEM_CACHE, pool); + if (!st->cache_file) { + return -1; + } + + result = apr_shm_create(&st->cache_shm, st->cache_bytes, st->cache_file, st->pool); if (result == EEXIST) { /* * The cache could have already been created (i.e. we may be a child process). See * if we can attach to the existing shared memory */ - result = apr_shm_attach(&util_ldap_shm, MODLDAP_SHMEM_CACHE, pool); + result = apr_shm_attach(&st->cache_shm, st->cache_file, st->pool); } if (result != APR_SUCCESS) { return result; } /* This will create a rmm "handler" to get into the shared memory area */ - apr_rmm_init(&util_ldap_rmm, NULL, - (void *)apr_shm_baseaddr_get(util_ldap_shm), reqsize, pool); + apr_rmm_init(&st->cache_rmm, NULL, (void *)apr_shm_baseaddr_get(st->cache_shm), st->cache_bytes, st->pool); #endif - apr_pool_cleanup_register(pool, NULL, util_ldap_cache_module_kill, apr_pool_cleanup_null); + apr_pool_cleanup_register(st->pool, st , util_ldap_cache_module_kill, apr_pool_cleanup_null); - util_ldap_cache = util_ald_create_cache(50, + st->util_ldap_cache = util_ald_create_cache(st, util_ldap_url_node_hash, util_ldap_url_node_compare, util_ldap_url_node_copy, Index: modules/experimental/util_ldap_cache.h =================================================================== RCS file: /home/cvspublic/httpd-2.0/modules/experimental/util_ldap_cache.h,v retrieving revision 1.6 diff -u -r1.6 util_ldap_cache.h --- modules/experimental/util_ldap_cache.h 21 Jan 2003 10:53:09 -0000 1.6 +++ modules/experimental/util_ldap_cache.h 12 Sep 2003 10:08:17 -0000 @@ -77,16 +77,18 @@ struct util_cache_node_t *next; } util_cache_node_t; -typedef struct util_ald_cache_t { - unsigned long size; /* Size of cache array */ - unsigned long maxentries; /* Maximum number of cache entries */ - unsigned long numentries; /* Current number of cache entries */ - unsigned long fullmark; /* Used to keep track of when cache becomes 3/4 full */ - apr_time_t marktime; /* Time that the cache became 3/4 full */ - unsigned long (*hash)(void *); /* Func to hash the payload */ - int (*compare)(void *, void *); /* Func to compare two payloads */ - void * (*copy)(void *); /* Func to alloc mem and copy payload to new mem */ - void (*free)(void *); /* Func to free mem used by the payload */ +typedef struct util_ald_cache util_ald_cache_t; + +struct util_ald_cache { + unsigned long size; /* Size of cache array */ + unsigned long maxentries; /* Maximum number of cache entries */ + unsigned long numentries; /* Current number of cache entries */ + unsigned long fullmark; /* Used to keep track of when cache becomes 3/4 full */ + apr_time_t marktime; /* Time that the cache became 3/4 full */ + unsigned long (*hash)(void *); /* Func to hash the payload */ + int (*compare)(void *, void *); /* Func to compare two payloads */ + void * (*copy)(util_ald_cache_t *cache, void *); /* Func to alloc mem and copy payload to new mem */ + void (*free)(util_ald_cache_t *cache, void *); /* Func to free mem used by the payload */ util_cache_node_t **nodes; unsigned long numpurges; /* No. of times the cache has been purged */ @@ -100,24 +102,23 @@ unsigned long hits; /* Number of cache hits */ unsigned long inserts; /* Number of inserts */ unsigned long removes; /* Number of removes */ -} util_ald_cache_t; #if APR_HAS_SHARED_MEMORY -apr_shm_t *util_ldap_shm; -apr_rmm_t *util_ldap_rmm; + apr_shm_t *shm_addr; + apr_rmm_t *rmm_addr; #endif -util_ald_cache_t *util_ldap_cache; + +}; #if APR_HAS_THREADS -apr_thread_rwlock_t *util_ldap_cache_lock; #define LDAP_CACHE_LOCK_CREATE(p) \ - if (!util_ldap_cache_lock) apr_thread_rwlock_create(&util_ldap_cache_lock, p) + if (!st->util_ldap_cache_lock) apr_thread_rwlock_create(&st->util_ldap_cache_lock, st->pool) #define LDAP_CACHE_WRLOCK() \ - apr_thread_rwlock_wrlock(util_ldap_cache_lock) + apr_thread_rwlock_wrlock(st->util_ldap_cache_lock) #define LDAP_CACHE_UNLOCK() \ - apr_thread_rwlock_unlock(util_ldap_cache_lock) + apr_thread_rwlock_unlock(st->util_ldap_cache_lock) #define LDAP_CACHE_RDLOCK() \ - apr_thread_rwlock_rdlock(util_ldap_cache_lock) + apr_thread_rwlock_rdlock(st->util_ldap_cache_lock) #else #define LDAP_CACHE_LOCK_CREATE(p) #define LDAP_CACHE_WRLOCK() @@ -192,35 +193,39 @@ /* util_ldap_cache.c */ unsigned long util_ldap_url_node_hash(void *n); int util_ldap_url_node_compare(void *a, void *b); -void *util_ldap_url_node_copy(void *c); -void util_ldap_url_node_free(void *n); +void *util_ldap_url_node_copy(util_ald_cache_t *cache, void *c); +void util_ldap_url_node_free(util_ald_cache_t *cache, void *n); unsigned long util_ldap_search_node_hash(void *n); int util_ldap_search_node_compare(void *a, void *b); -void *util_ldap_search_node_copy(void *c); -void util_ldap_search_node_free(void *n); +void *util_ldap_search_node_copy(util_ald_cache_t *cache, void *c); +void util_ldap_search_node_free(util_ald_cache_t *cache, void *n); unsigned long util_ldap_compare_node_hash(void *n); int util_ldap_compare_node_compare(void *a, void *b); -void *util_ldap_compare_node_copy(void *c); -void util_ldap_compare_node_free(void *n); +void *util_ldap_compare_node_copy(util_ald_cache_t *cache, void *c); +void util_ldap_compare_node_free(util_ald_cache_t *cache, void *n); unsigned long util_ldap_dn_compare_node_hash(void *n); int util_ldap_dn_compare_node_compare(void *a, void *b); -void *util_ldap_dn_compare_node_copy(void *c); -void util_ldap_dn_compare_node_free(void *n); +void *util_ldap_dn_compare_node_copy(util_ald_cache_t *cache, void *c); +void util_ldap_dn_compare_node_free(util_ald_cache_t *cache, void *n); /* util_ldap_cache_mgr.c */ -void util_ald_free(const void *ptr); -void *util_ald_alloc(unsigned long size); -const char *util_ald_strdup(const char *s); +/* Cache alloc and free function, dealing or not with shm */ +void util_ald_free(apr_rmm_t *rmm_addr, const void *ptr); +void *util_ald_alloc(apr_rmm_t *rmm_addr, unsigned long size); +const char *util_ald_strdup(apr_rmm_t *rmm_addr, const char *s); + +/* Cache managing function */ unsigned long util_ald_hash_string(int nstr, ...); void util_ald_cache_purge(util_ald_cache_t *cache); util_url_node_t *util_ald_create_caches(util_ldap_state_t *s, const char *url); -util_ald_cache_t *util_ald_create_cache(unsigned long maxentries, +util_ald_cache_t *util_ald_create_cache(util_ldap_state_t *st, unsigned long (*hashfunc)(void *), int (*comparefunc)(void *, void *), - void * (*copyfunc)(void *), - void (*freefunc)(void *)); + void * (*copyfunc)(util_ald_cache_t *cache, void *), + void (*freefunc)(util_ald_cache_t *cache, void *)); + void util_ald_destroy_cache(util_ald_cache_t *cache); void *util_ald_cache_fetch(util_ald_cache_t *cache, void *payload); void util_ald_cache_insert(util_ald_cache_t *cache, void *payload); Index: modules/experimental/util_ldap_cache_mgr.c =================================================================== RCS file: /home/cvspublic/httpd-2.0/modules/experimental/util_ldap_cache_mgr.c,v retrieving revision 1.5 diff -u -r1.5 util_ldap_cache_mgr.c --- modules/experimental/util_ldap_cache_mgr.c 21 Jan 2003 10:53:09 -0000 1.5 +++ modules/experimental/util_ldap_cache_mgr.c 12 Sep 2003 10:08:18 -0000 @@ -112,14 +112,16 @@ 0 }; -void util_ald_free(const void *ptr) +void util_ald_free(apr_rmm_t *rmm_addr, const void *ptr) { #if APR_HAS_SHARED_MEMORY - if (util_ldap_shm) { + if (rmm_addr) { if (ptr) - apr_rmm_free(util_ldap_rmm, apr_rmm_offset_get(util_ldap_rmm, (void *)ptr)); + /* Free in shared memory */ + apr_rmm_free(rmm_addr, apr_rmm_offset_get(rmm_addr, (void *)ptr)); } else { if (ptr) + /* Cache shm is not used */ free((void *)ptr); } #else @@ -128,14 +130,16 @@ #endif } -void *util_ald_alloc(unsigned long size) +void *util_ald_alloc(apr_rmm_t *rmm_addr, unsigned long size) { if (0 == size) return NULL; #if APR_HAS_SHARED_MEMORY - if (util_ldap_shm) { - return (void *)apr_rmm_addr_get(util_ldap_rmm, apr_rmm_calloc(util_ldap_rmm, size)); + if (rmm_addr) { + /* allocate from shared memory */ + return (void *)apr_rmm_addr_get(rmm_addr, apr_rmm_calloc(rmm_addr, size)); } else { + /* Cache shm is not used */ return (void *)calloc(sizeof(char), size); } #else @@ -143,11 +147,12 @@ #endif } -const char *util_ald_strdup(const char *s) +const char *util_ald_strdup(apr_rmm_t *rmm_addr, const char *s) { #if APR_HAS_SHARED_MEMORY - if (util_ldap_shm) { - char *buf = (char *)apr_rmm_addr_get(util_ldap_rmm, apr_rmm_calloc(util_ldap_rmm, strlen(s)+1)); + if (rmm_addr) { + /* allocate from shared memory */ + char *buf = (char *)apr_rmm_addr_get(rmm_addr, apr_rmm_calloc(rmm_addr, strlen(s)+1)); if (buf) { strcpy(buf, s); return buf; @@ -156,6 +161,7 @@ return NULL; } } else { + /* Cache shm is not used */ return strdup(s); } #else @@ -217,8 +223,8 @@ while (p != NULL) { if (p->add_time < cache->marktime) { q = p->next; - (*cache->free)(p->payload); - util_ald_free(p); + (*cache->free)(cache, p->payload); + util_ald_free(cache->rmm_addr, p); cache->numentries--; cache->npurged++; p = q; @@ -247,17 +253,17 @@ util_ald_cache_t *dn_compare_cache; /* create the three caches */ - search_cache = util_ald_create_cache(st->search_cache_size, + search_cache = util_ald_create_cache(st, util_ldap_search_node_hash, util_ldap_search_node_compare, util_ldap_search_node_copy, util_ldap_search_node_free); - compare_cache = util_ald_create_cache(st->compare_cache_size, + compare_cache = util_ald_create_cache(st, util_ldap_compare_node_hash, util_ldap_compare_node_compare, util_ldap_compare_node_copy, util_ldap_compare_node_free); - dn_compare_cache = util_ald_create_cache(st->compare_cache_size, + dn_compare_cache = util_ald_create_cache(st, util_ldap_dn_compare_node_hash, util_ldap_dn_compare_node_compare, util_ldap_dn_compare_node_copy, @@ -272,7 +278,7 @@ curl->compare_cache = compare_cache; curl->dn_compare_cache = dn_compare_cache; - util_ald_cache_insert(util_ldap_cache, curl); + util_ald_cache_insert(st->util_ldap_cache, curl); } @@ -280,32 +286,32 @@ } -util_ald_cache_t *util_ald_create_cache(unsigned long maxentries, +util_ald_cache_t *util_ald_create_cache(util_ldap_state_t *st, unsigned long (*hashfunc)(void *), int (*comparefunc)(void *, void *), - void * (*copyfunc)(void *), - void (*freefunc)(void *)) + void * (*copyfunc)(util_ald_cache_t *cache, void *), + void (*freefunc)(util_ald_cache_t *cache, void *)) { util_ald_cache_t *cache; unsigned long i; - if (maxentries <= 0) + if (st->search_cache_size <= 0) return NULL; - cache = (util_ald_cache_t *)util_ald_alloc(sizeof(util_ald_cache_t)); + cache = (util_ald_cache_t *)util_ald_alloc(st->cache_rmm, sizeof(util_ald_cache_t)); if (!cache) return NULL; - cache->maxentries = maxentries; + cache->maxentries = st->search_cache_size; cache->numentries = 0; - cache->size = maxentries / 3; + cache->size = st->search_cache_size / 3; if (cache->size < 64) cache->size = 64; for (i = 0; primes[i] && primes[i] < cache->size; ++i) ; cache->size = primes[i]? primes[i] : primes[i-1]; - cache->nodes = (util_cache_node_t **)util_ald_alloc(cache->size * sizeof(util_cache_node_t *)); + cache->nodes = (util_cache_node_t **)util_ald_alloc(cache->rmm_addr, cache->size * sizeof(util_cache_node_t *)); if (!cache->nodes) { - util_ald_free(cache); + util_ald_free(cache->rmm_addr, cache); return NULL; } @@ -345,13 +351,13 @@ q = NULL; while (p != NULL) { q = p->next; - (*cache->free)(p->payload); - util_ald_free(p); + (*cache->free)(cache, p->payload); + util_ald_free(cache->rmm_addr, p); p = q; } } - util_ald_free(cache->nodes); - util_ald_free(cache); + util_ald_free(cache->rmm_addr, cache->nodes); + util_ald_free(cache->rmm_addr, cache); } void *util_ald_cache_fetch(util_ald_cache_t *cache, void *payload) @@ -392,9 +398,9 @@ cache->inserts++; hashval = (*cache->hash)(payload) % cache->size; - node = (util_cache_node_t *)util_ald_alloc(sizeof(util_cache_node_t)); + node = (util_cache_node_t *)util_ald_alloc(cache->rmm_addr, sizeof(util_cache_node_t)); node->add_time = apr_time_now(); - node->payload = (*cache->copy)(payload); + node->payload = (*cache->copy)(cache, payload); node->next = cache->nodes[hashval]; cache->nodes[hashval] = node; if (++cache->numentries == cache->fullmark) @@ -431,8 +437,8 @@ /* We found the node and it's not the first in the list */ q->next = p->next; } - (*cache->free)(p->payload); - util_ald_free(p); + (*cache->free)(cache, p->payload); + util_ald_free(cache->rmm_addr, p); cache->numentries--; } @@ -499,16 +505,19 @@ return buf; } -char *util_ald_cache_display(apr_pool_t *pool) +char *util_ald_cache_display(apr_pool_t *pool, util_ldap_state_t *st) { unsigned long i; char *buf, *t1, *t2, *t3; + util_ald_cache_t *util_ldap_cache = st->util_ldap_cache; + + if (!util_ldap_cache) { return "Cache has not been enabled/initialised."; } - buf = util_ald_cache_display_stats(pool, util_ldap_cache, "LDAP URL Cache"); + buf = util_ald_cache_display_stats(pool, st->util_ldap_cache, "LDAP URL Cache"); for (i=0; i < util_ldap_cache->size; ++i) { util_cache_node_t *p; --=-hTwz1xuPwY44dTaCk1Kj--