Return-Path: Delivered-To: apmail-apr-dev-archive@apr.apache.org Received: (qmail 37126 invoked by uid 500); 13 Sep 2001 03:45:00 -0000 Mailing-List: contact dev-help@apr.apache.org; run by ezmlm Precedence: bulk List-Post: List-Help: List-Unsubscribe: List-Subscribe: Delivered-To: mailing list dev@apr.apache.org Received: (qmail 37096 invoked from network); 13 Sep 2001 03:44:59 -0000 Date: Wed, 12 Sep 2001 23:45:11 -0400 Message-Id: <200109130345.XAA16937@Mail.MeepZor.Com> From: Rodent of Unusual Size To: APR developers Subject: [STATUS] (apr) Wed Sep 12 23:45:11 EDT 2001 X-Note: This is an automated message. X-Spam-Rating: daedalus.apache.org 1.6.2 0/1000/N APACHE PORTABLE RUNTIME (APR) LIBRARY STATUS: -*-text-*- Last modified at [$Date: 2001/09/11 01:06:13 $] Release: 2.0a9 : released December 12, 2000 2.0a8 : released November 20, 2000 2.0a7 : released October 8, 2000 2.0a6 : released August 18, 2000 2.0a5 : released August 4, 2000 2.0a4 : released June 7, 2000 2.0a3 : released April 28, 2000 2.0a2 : released March 31, 2000 2.0a1 : released March 10, 2000 RELEASE SHOWSTOPPERS: * complete the efforts started by DougM for cleaner fn naming conventions: see proposed name changes in renames_pending and offer up any additions/vetos/clarifications. DougM offered to complete the work with his nifty perl rename script at the hackathon. * When Win32 apr_proc_create was fixed, the apr_proc_t hproc member was added for that platform. That's a problem (and was when pid was abused as well) since nobody goes and cleans up hproc after the process is dead. Can't do a pool cleanup, since apr_proc_create didn't allocate the apr_proc_t storage. (Aren't transparent types swell?) Suggestions? RELEASE NON-SHOWSTOPPERS BUT WOULD BE REAL NICE TO WRAP THESE UP: * Get OTHER_CHILD support into Win32 Status: Bill S. is looking into this * Win32 apr_proc_create fails to create 16 bit apps detached (a win32 bug.) The question - test in advance (slow) or recover gracefully from failure and try again? Only the test method will work on Win9x, since it will appear to work, only to encounter mangled pipes. Win2K (NT?) simply fails. * SysV semaphore support isn't usable by Apache when started as root because we don't have a way to allow the semaphore to be used by the configured User and Group. Current work-around: change the initial permissions to 0666. Needed code: See 1.3's http_main.c, SysV sem flavor of accept_mutex_init(). Status: Jim will look into this * Build scripts do not recognise AIX 4.2.1 pthreads Justin says: "Is this still true?" * Win32: Implement apr_shm_ functions Status: rbb insists he has thoughts about splitting apr_shm_* mechanisms to support muliple models (some sort of 'keyed' schema as well as anonymous inheritable shmem), and has a possible solution to the 'ask for 1MB, then ask for 4x256kb bogosity, so we are waiting on this. Justin says: "That problem should be fixed now because we ignore memory management with shared memory on Unix (at least). So, the Win32 guys should be able to go ahead if they want." * FirstBill says we need a new procattr, APR_CREATE_SUSPENDED (or something similar) to direct ap_create_process to create the process suspended. We also need a call to wake up the suspended process. This may not be able to be implemented everywhere though. Status: OtherBill asks, why? What is the benefit, how is it portably implemented? Unless this creates some tangible that mirrors another platform, then I'm -1. * Replace tables with a proper opaque ADT that has pluggable implementations (including something like the existing data type, plus hash tables for speed, with options for more later). Status: fanf is working on this. * add a version number to apr_initialize() as an extra failsafe against (APR) library version skew. MsgID: Status: Greg +1 (volunteers), Jeff +1, Ryan +1, Tony -0(?), david +1 * add apr_crypt() and APR_HAS_CRYPT for apps to determine whether the crypt() function is available, and a way to call it (whether it is located in libc, libcrypt, or libufc) Justin says: Should apr_crypt() be in apr-util? Status: Greg +1 (volunteers) * apr_create_lock() changes: - It ignores the "type" parameter, so toss it. - The fname param is allowed to be NULL on the Unix platform. Change it to always use the passed value, and check callers. rbb says: The type parameter is supposed to be used to determine if we are working with a read/write lock or a mutex. The fname parameter is essentially required if you want to be portable, but I dislike wasting cycles to outsmart the programmer. Justin says: The type parameter is now used. Status: david +1 rbb -1 * configure.in does post-processing on the AC_OUTPUT files (for VPATH support). This means that config.status doesn't do the right thing when you re-run it. We ought to revamp the makefiles to do the right AC_SUBST stuff rather than depend upon rewriting. Sascha: As the rewriter is a crude hack, I would not worry too much about it. It is designed to go away once we have a proper build system in place. One of the perceived deficiencies of automake is that it uses AC_SUBST too often, thereby slowing down the task of generating Makefiles significantly, because it applies dozens of substitutions to each Makefile. And why? Make's built-in macro processing is much more powerful, and combined with the include facility, generating Makefiles becomes simpler and faster. Justin says: "I think this got fixed with Roy's build changes." * use os_(un)cork in network_io/unix/sendrecv.c for FreeBSD's sendfile implementation. david: The socket options stuff is now in and using it should reduce the number of syscalls that are required for os_cork and uncork, so the code should be reviewed to make use of the new calls. If no-one beats me to it I'll get around to it soonish... * toss the per-Makefile setup of INCLUDES; shift to rules.mk.in rbb: This is a bad thing IMHO. If we do this, then we can't use these makefiles for anything else. For example, apr-util * add the rest of the pool accessor declare/impl macros. Justin says: Both thread and file have the accessors now. Any others? Status: Greg volunteers * I think apr_open_stderr() and friends dup() the descriptor. That would allow the new/returned file to be closed (via pool cleanup or manually) without accidentally closing stderr/out. Justin says: Is this "I think it should?" * need to export the shared library extension (e.g. ".so") for the platform. clients need to use this to construct filenames to pass to apr_dso_load() -- note on Win32 we distinguish 'apache module' names from other 'loadable module' names, so be careful with Apache's directive. * APR memory code - code has been added but we still need to - decide on a better name for the code - reformat to APR style (think this is now done, but some tabs left) - test on more systems - add more detailed tests to testmem.c Status: Optionally enable it with --enable-sms. Still wildly unproven. But, it actually works as a replacement for pools now. It may even not segfault when running httpd under high-loads. The performance impact/benefit still needs to be examined. * In line with the new SMS code is the fact that threading and pools just are not working together well. This is due to the fact that the pool code has one global mutex (alloc_mutex) and one freelist (block_freelist) for all pools to share. This means that only one worker can be allocating memory at any given time. This is probably the reason why Apache 2.0 is faster with prefork MPM (thread-disabled APR) than threaded MPM. The solution to this is most likely to incorporate a rework of the pools to use the new SMS code and allow certain pools (i.e. request pools in httpd-2.0) to have an option for no locking (as they can't have contention by definition). This would mean that the mutex and freelist must be moved inside of apr_pool_t. Therefore, this is the jumping-off point into SMS. Justin: The SMS code has been checked into CVS (see above). To solve this problem, we want only one trivial SMS per thread which acts as the parent for all SMSs in that thread (giving us thread-local allocation). Each descendant SMS should be something along the lines of a tracking SMS. That's how I see it anyway. There are other possibilities. Any of those probably work as well. See the apr archives for more info. We're still debating this. * Possible gmtime_r replacement in explode_time On Solaris (and possibly others), the gmtime_r libc function obtains a mutex. We have seen 21/25 threads being blocked in this mutex on a threaded httpd MPM when requesting static pages. It may be worth it to hand optimize this since there is no real need for a mutex at the system level (straight arithmetic from what I can tell). If you have access to the Solaris source code: osnet_volume/usr/src/lib/libc/port/gen/time_comm.c. * Add a way to query APR for what features it has at runtime (i.e. threads). Justin says: I'm not completely sold on this, but it has been mentioned before and at least added to STATUS. * apr_xlate.h generates a bunch of compiler warnings. * fcntl() oddness on Solaris. Under high loads, fcntl() decides to return error code 46 (ENOLCK). httpd (prefork MPM) error log says (predictably): (46)No record locks available: couldn't grab the accept mutex All of the children report this and subsequently exits. httpd is now hosed. AFAICT, this does not look to be an out-of-fds error. Solaris's man page says: ENOLCK The cmd argument is F_SETLK, F_SETLK64, F_SETLKW, or F_SETLKW64 and satisfying the lock or unlock request would result in the number of locked regions in the system exceeding a system-imposed limit. Justin says: What is this system-imposed limit and how do we change it? This gives me more rationale for switching the default interprocess lock mechanism to pthread (if available). * Generate a good bug report to send to the FreeBSD hackers that details the problems we have seen with threads and system calls (specifically sendfile data is corrupted). From our analysis so far, we don't think that this is an APR issue, but rather a FreeBSD kernel issue. Our current solution is to just disable threads across the board on FreeBSD. MsgID: <20010828091959.D17570@ebuilt.com> Documentation that needs writing: * API documentation Ian Says: APR Stuff in now in Doxygen format, which is the first step. Stuff waiting for code thawing after Beta 1: * Identify and implement those protection bits that have general usefulness, perhaps hidden, generic read-only [immutable], effective current user permissions, etc. APR Stackable Memory Code ========================= This is just a small list of things yet to be done, or things that we may want/need to consider. - add a shared memory module. - locking needs to be addressed. The scope of the locks needs to be defined and it's likely we'll need some way of varying the scope when locking. - given the problems that can occur when trying to find alloc/free problems we should probably have a special debug memory system that records everything it does and any other information we think is useful. - in addition to the debugging system, we need to look at methods of checking memory allocations to ensure we're behaving when we have the ASSERT_MEMORY flag turned on. The pools in 1.3 had code from dean and Roy, Greg has added some special stuff for pools under Linux on 2.0, so we just need some ideas