Return-Path: Delivered-To: apmail-apr-dev-archive@www.apache.org Received: (qmail 39044 invoked from network); 6 May 2008 06:44:40 -0000 Received: from hermes.apache.org (HELO mail.apache.org) (140.211.11.2) by minotaur.apache.org with SMTP; 6 May 2008 06:44:40 -0000 Received: (qmail 1473 invoked by uid 500); 6 May 2008 06:44:41 -0000 Delivered-To: apmail-apr-dev-archive@apr.apache.org Received: (qmail 1082 invoked by uid 500); 6 May 2008 06:44:41 -0000 Mailing-List: contact dev-help@apr.apache.org; run by ezmlm Precedence: bulk List-Post: List-Help: List-Unsubscribe: List-Id: Delivered-To: mailing list dev@apr.apache.org Received: (qmail 1070 invoked by uid 99); 6 May 2008 06:44:41 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 05 May 2008 23:44:41 -0700 X-ASF-Spam-Status: No, hits=-0.0 required=10.0 tests=SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (athena.apache.org: domain of chip@force-elite.com designates 72.232.80.58 as permitted sender) Received: from [72.232.80.58] (HELO constant.northnitch.com) (72.232.80.58) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 06 May 2008 06:43:54 +0000 Received: from BlueWater.local (c-67-164-104-143.hsd1.ca.comcast.net [67.164.104.143]) by constant.northnitch.com (Postfix) with ESMTP id 014B98733; Tue, 6 May 2008 01:44:03 -0500 (CDT) Message-ID: <481FFE33.9010504@force-elite.com> Date: Mon, 05 May 2008 23:44:03 -0700 From: Paul Querna User-Agent: Thunderbird 2.0.0.14 (Macintosh/20080421) MIME-Version: 1.0 To: dev@httpd.apache.org CC: Theo Schlossnagle , APR Developer List Subject: Re: [PATCH] DTrace probes patch. References: <6AFBCFE8-4CCA-4A02-8A43-F9170689695D@omniti.com> In-Reply-To: <6AFBCFE8-4CCA-4A02-8A43-F9170689695D@omniti.com> X-Enigmail-Version: 0.95.6 Content-Type: multipart/mixed; boundary="------------030909060101030802090909" X-Virus-Checked: Checked by ClamAV on apache.org This is a multi-part message in MIME format. --------------030909060101030802090909 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Theo Schlossnagle wrote: > Hello all, > > The probes can really give a different perspective on production > environments. > > The patch has some nastiness to it that I'm sure people would want to > strategize on cleaning up. The main issue being that Apache is > constructed from a bunch of static apr/libtool built libraries. DTrace > doesn't work on archives. So, I've got some bloody knuckles from > bending the build system to keep things as normal ELF objects. > > I had a first good step... and then a red herring issue that I worked > through with the DTrace team led me to a much less-elegant way of > building. I could revert to the original process (ld -r -o the objects > into library-esque packages) as DTrace can work on those. > > The probes are neatly defined and placed, but the patches to the build > system are gruesome. Uhm. yah. libtool-dep-extract, doesn't work with jlibtool.[1] So, to get this to compile on OSX, I found out dtrace on OSX doesn't have -G, and Apple says headers only/-h is enough. [2] Not having to muck with the prelink steps makes this whole thing much easier to deal with. For trunk, I had to mess around with http_request.c, as it seems the 2.2.x version is quite out of sync with trunk. (All due to the async request processing work that happened in trunk). The install-dtrace target is also slightly... worrying. For example: @chown -R root:bin $(DESTDIR)/usr/lib Seems like a bad thing for non-solaris operating systems, like OSX :-) Attached is a patch for todays trunk, which seems to compile on OSX 10.5. To get this in, I think we should make include/apache_probes.h a target in the makefile, and not build it in configure.in. I'll try to get something workong on OSX tomorrow and then hopefully someone can help me with Solaris later, > The apr-util patch to the apr_hooks.h is simple and affords some nice > probability for future probe uses. I've committed this patch to ARP-Util in r653681. [3] I think we should add --enable-dtrace to APR{,-Util}'s configure.in, so that we can wire up other things, and have a standard way of adding things to the CPPFLAGS to enable it. (And to append those CPPFLAGS to apr/apr-util's -config programs) -Paul [1] - http://svn.apache.org/repos/asf/apr/apr/trunk/build/jlibtool.c [2] - http://archive.netbsd.se/?ml=dtrace-discuss&a=2007-10&m=5528648 [3] - https://svn.apache.org/viewvc?view=rev&revision=653681 --------------030909060101030802090909 Content-Type: text/plain; x-mac-type="0"; x-mac-creator="0"; name="httpd-dtrace-trunk.patch" Content-Transfer-Encoding: 7bit Content-Disposition: inline; filename="httpd-dtrace-trunk.patch" Index: Makefile.in =================================================================== --- Makefile.in (revision 653681) +++ Makefile.in (working copy) @@ -16,7 +16,7 @@ TARGETS = $(PROGRAMS) $(shared_build) $(other_targets) INSTALL_TARGETS = install-conf install-htdocs install-error install-icons \ install-other install-cgi install-include install-suexec install-build \ - install-man + install-man install-dtrace DISTCLEAN_TARGETS = include/ap_config_auto.h include/ap_config_layout.h \ modules.c config.cache config.log config.status build/config_vars.mk \ @@ -28,6 +28,12 @@ include $(top_builddir)/build/rules.mk include $(top_srcdir)/build/program.mk +install-dtrace: + @echo Installing DTrace library + @$(MKINSTALLDIRS) $(DESTDIR)/usr/lib/dtrace + @cp $(top_srcdir)/ap.d $(DESTDIR)/usr/lib/dtrace + @chmod 0644 $(DESTDIR)/usr/lib/dtrace/ap.d + install-conf: @echo Installing configuration files @$(MKINSTALLDIRS) $(DESTDIR)$(sysconfdir) $(DESTDIR)$(sysconfdir)/extra Index: server/Makefile.in =================================================================== --- server/Makefile.in (revision 653681) +++ server/Makefile.in (working copy) @@ -61,7 +61,7 @@ for dir in $(EXPORT_DIRS_APR); do \ (ls $$dir/ap[ru].h $$dir/ap[ru]_*.h >> $$tmp 2>/dev/null); \ done; \ - sort -u $$tmp > $@; \ + sort -u $$tmp | grep -v apache_probes.h > $@; \ rm -f $$tmp exports.c: export_files Index: server/protocol.c =================================================================== --- server/protocol.c (revision 653681) +++ server/protocol.c (working copy) @@ -842,9 +842,11 @@ apr_socket_t *csd; apr_interval_time_t cur_timeout; + apr_pool_create(&p, conn->pool); apr_pool_tag(p, "request"); r = apr_pcalloc(p, sizeof(request_rec)); + AP_READ_REQUEST_ENTRY((intptr_t)r, (uintptr_t)conn); r->pool = p; r->connection = conn; r->server = conn->base_server; @@ -894,11 +896,12 @@ ap_update_child_status(conn->sbh, SERVER_BUSY_LOG, r); ap_run_log_transaction(r); apr_brigade_destroy(tmp_bb); - return r; + goto traceout; } apr_brigade_destroy(tmp_bb); - return NULL; + r = NULL; + goto traceout; } /* We may have been in keep_alive_timeout mode, so toggle back @@ -921,7 +924,7 @@ ap_update_child_status(conn->sbh, SERVER_BUSY_LOG, r); ap_run_log_transaction(r); apr_brigade_destroy(tmp_bb); - return r; + goto traceout; } if (apr_table_get(r->headers_in, "Transfer-Encoding") @@ -949,7 +952,7 @@ ap_update_child_status(conn->sbh, SERVER_BUSY_LOG, r); ap_run_log_transaction(r); apr_brigade_destroy(tmp_bb); - return r; + goto traceout; } } @@ -1003,14 +1006,15 @@ ap_send_error_response(r, 0); ap_update_child_status(conn->sbh, SERVER_BUSY_LOG, r); ap_run_log_transaction(r); - return r; + goto traceout; } if ((access_status = ap_run_post_read_request(r))) { ap_die(access_status, r); ap_update_child_status(conn->sbh, SERVER_BUSY_LOG, r); ap_run_log_transaction(r); - return NULL; + r = NULL; + goto traceout; } if (((expect = apr_table_get(r->headers_in, "Expect")) != NULL) @@ -1032,11 +1036,16 @@ ap_send_error_response(r, 0); ap_update_child_status(conn->sbh, SERVER_BUSY_LOG, r); ap_run_log_transaction(r); - return r; + goto traceout; } } + AP_READ_REQUEST_SUCCESS((uintptr_t)r, (char *)r->method, (char *)r->uri, + (char *)r->server->defn_name, r->status); return r; + traceout: + AP_READ_REQUEST_FAILURE((uintptr_t)r); + return r; } /* if a request with a body creates a subrequest, clone the original request's Index: modules/http/http_request.c =================================================================== --- modules/http/http_request.c (revision 653681) +++ modules/http/http_request.c (working copy) @@ -234,6 +234,7 @@ * Use this hook with extreme care and only if you know what you are * doing. */ + AP_PROCESS_REQUEST_ENTRY((uintptr_t)r, r->uri); if (ap_extended_status) ap_time_process_request(r->connection->sbh, START_PREQUEST); access_status = ap_run_quick_handler(r, 0); /* Not a look-up request */ @@ -273,8 +274,11 @@ c->cs->state = CONN_STATE_WRITE_COMPLETION; check_pipeline(c); - if (ap_extended_status) + if (ap_extended_status) { ap_time_process_request(c->sbh, STOP_PREQUEST); + } + + AP_PROCESS_REQUEST_RETURN((uintptr_t)r, r->uri, access_status); } void ap_process_request(request_rec *r) @@ -305,11 +309,13 @@ " client", r->unparsed_uri); } } + if (ap_extended_status) { ap_time_process_request(c->sbh, STOP_PREQUEST); } } + static apr_table_t *rename_original_env(apr_pool_t *p, apr_table_t *t) { const apr_array_header_t *env_arr = apr_table_elts(t); @@ -489,6 +495,8 @@ request_rec *new = internal_internal_redirect(new_uri, r); int access_status; + AP_INTERNAL_REDIRECT(r->uri, new_uri); + /* ap_die was already called, if an error occured */ if (!new) { return; Index: modules/proxy/mod_proxy.c =================================================================== --- modules/proxy/mod_proxy.c (revision 653681) +++ modules/proxy/mod_proxy.c (working copy) @@ -979,6 +979,7 @@ ap_log_error(APLOG_MARK, APLOG_DEBUG, 0, r->server, "Running scheme %s handler (attempt %d)", scheme, attempts); + AP_PROXY_RUN(r); access_status = proxy_run_scheme_handler(r, worker, conf, url, NULL, 0); if (access_status == OK) @@ -1026,6 +1027,7 @@ ap_proxy_post_request(worker, balancer, r, conf); proxy_run_request_status(&access_status, r); + AP_PROXY_RUN_FINISHED(r, attempts, access_status); return access_status; } Index: modules/mappers/mod_rewrite.c =================================================================== --- modules/mappers/mod_rewrite.c (revision 653681) +++ modules/mappers/mod_rewrite.c (working copy) @@ -486,7 +486,8 @@ conf = ap_get_module_config(r->server->module_config, &rewrite_module); - if (!conf->rewritelogfp || level > conf->rewriteloglevel) { + if ((!conf->rewritelogfp || level > conf->rewriteloglevel) && + !AP_REWRITE_LOG_ENABLED()) { return; } @@ -520,6 +521,11 @@ perdir ? "] ": "", text); + AP_REWRITE_LOG((uintptr_t)r, level, r->main ? 0 : 1, (char *)ap_get_server_name(r), logline); + + if (!conf->rewritelogfp || level > conf->rewriteloglevel) + return; + rv = apr_global_mutex_lock(rewrite_log_lock); if (rv != APR_SUCCESS) { ap_log_rerror(APLOG_MARK, APLOG_ERR, rv, r, Index: apache_probes.d =================================================================== --- apache_probes.d (revision 0) +++ apache_probes.d (revision 0) @@ -0,0 +1,172 @@ +provider ap { + /* Explicit, core */ + probe internal__redirect(char *, char *); + probe process__request__entry(unsigned long, char *); + probe process__request__return(unsigned long, char *, unsigned int); + probe read__request__entry(unsigned long, unsigned long); + probe read__request__success(unsigned long, char *, char *, char *, unsigned int); + probe read__request__failure(unsigned long); + + /* Explicit, modules */ + probe rewrite__log(unsigned long, int, int, char *, char *); + probe proxy__run(unsigned long); + probe proxy__run__finished(unsigned long, unsigned int, unsigned int); + + /* Implicit, APR hooks */ + probe translate_name__entry(); + probe translate_name__dispatch__invoke(char *); + probe translate_name__dispatch__complete(char *, unsigned int); + probe translate_name__return(unsigned int); + probe map_to_storage__entry(); + probe map_to_storage__dispatch__invoke(char *); + probe map_to_storage__dispatch__complete(char *, unsigned int); + probe map_to_storage__return(unsigned int); + probe check_user_id__entry(); + probe check_user_id__dispatch__invoke(char *); + probe check_user_id__dispatch__complete(char *, unsigned int); + probe check_user_id__return(unsigned int); + probe fixups__entry(); + probe fixups__dispatch__invoke(char *); + probe fixups__dispatch__complete(char *, unsigned int); + probe fixups__return(unsigned int); + probe type_checker__entry(); + probe type_checker__dispatch__invoke(char *); + probe type_checker__dispatch__complete(char *, unsigned int); + probe type_checker__return(unsigned int); + probe access_checker__entry(); + probe access_checker__dispatch__invoke(char *); + probe access_checker__dispatch__complete(char *, unsigned int); + probe access_checker__return(unsigned int); + probe auth_checker__entry(); + probe auth_checker__dispatch__invoke(char *); + probe auth_checker__dispatch__complete(char *, unsigned int); + probe auth_checker__return(unsigned int); + probe insert_filter__entry(); + probe insert_filter__dispatch__invoke(char *); + probe insert_filter__dispatch__complete(char *, unsigned int); + probe insert_filter__return(unsigned int); + probe create_connection__entry(); + probe create_connection__dispatch__invoke(char *); + probe create_connection__dispatch__complete(char *, unsigned int); + probe create_connection__return(unsigned int); + probe pre_connection__entry(); + probe pre_connection__dispatch__invoke(char *); + probe pre_connection__dispatch__complete(char *, unsigned int); + probe pre_connection__return(unsigned int); + probe process_connection__entry(); + probe process_connection__dispatch__invoke(char *); + probe process_connection__dispatch__complete(char *, unsigned int); + probe process_connection__return(unsigned int); + probe create_request__entry(); + probe create_request__dispatch__invoke(char *); + probe create_request__dispatch__complete(char *, unsigned int); + probe create_request__return(unsigned int); + probe header_parser__entry(); + probe header_parser__dispatch__invoke(char *); + probe header_parser__dispatch__complete(char *, unsigned int); + probe header_parser__return(unsigned int); + probe pre_config__entry(); + probe pre_config__dispatch__invoke(char *); + probe pre_config__dispatch__complete(char *, unsigned int); + probe pre_config__return(unsigned int); + probe test_config__entry(); + probe test_config__dispatch__invoke(char *); + probe test_config__dispatch__complete(char *, unsigned int); + probe test_config__return(unsigned int); + probe post_config__entry(); + probe post_config__dispatch__invoke(char *); + probe post_config__dispatch__complete(char *, unsigned int); + probe post_config__return(unsigned int); + probe open_logs__entry(); + probe open_logs__dispatch__invoke(char *); + probe open_logs__dispatch__complete(char *, unsigned int); + probe open_logs__return(unsigned int); + probe child_init__entry(); + probe child_init__dispatch__invoke(char *); + probe child_init__dispatch__complete(char *, unsigned int); + probe child_init__return(unsigned int); + probe handler__entry(); + probe handler__dispatch__invoke(char *); + probe handler__dispatch__complete(char *, unsigned int); + probe handler__return(unsigned int); + probe quick_handler__entry(); + probe quick_handler__dispatch__invoke(char *); + probe quick_handler__dispatch__complete(char *, unsigned int); + probe quick_handler__return(unsigned int); + probe optional_fn_retrieve__entry(); + probe optional_fn_retrieve__dispatch__invoke(char *); + probe optional_fn_retrieve__dispatch__complete(char *, unsigned int); + probe optional_fn_retrieve__return(unsigned int); + probe get_mgmt_items__entry(); + probe get_mgmt_items__dispatch__invoke(char *); + probe get_mgmt_items__dispatch__complete(char *, unsigned int); + probe get_mgmt_items__return(unsigned int); + probe pre_mpm__entry(); + probe pre_mpm__dispatch__invoke(char *); + probe pre_mpm__dispatch__complete(char *, unsigned int); + probe pre_mpm__return(unsigned int); + probe default_port__entry(); + probe default_port__dispatch__invoke(char *); + probe default_port__dispatch__complete(char *, unsigned int); + probe default_port__return(unsigned int); + probe http_scheme__entry(); + probe http_scheme__dispatch__invoke(char *); + probe http_scheme__dispatch__complete(char *, unsigned int); + probe http_scheme__return(unsigned int); + probe error_log__entry(); + probe error_log__dispatch__invoke(char *); + probe error_log__dispatch__complete(char *, unsigned int); + probe error_log__return(unsigned int); + probe log_transaction__entry(); + probe log_transaction__dispatch__invoke(char *); + probe log_transaction__dispatch__complete(char *, unsigned int); + probe log_transaction__return(unsigned int); + probe monitor__entry(); + probe monitor__dispatch__invoke(char *); + probe monitor__dispatch__complete(char *, unsigned int); + probe monitor__return(unsigned int); + probe post_read_request__entry(); + probe post_read_request__dispatch__invoke(char *); + probe post_read_request__dispatch__complete(char *, unsigned int); + probe post_read_request__return(unsigned int); + probe get_suexec_identity__entry(); + probe get_suexec_identity__dispatch__invoke(char *); + probe get_suexec_identity__dispatch__complete(char *, unsigned int); + probe get_suexec_identity__return(unsigned int); + probe insert_error_filter__entry(); + probe insert_error_filter__dispatch__invoke(char *); + probe insert_error_filter__dispatch__complete(char *, unsigned int); + probe insert_error_filter__return(unsigned int); + + /* Implicit, APR hooks for proxy */ + probe scheme_handler__entry(); + probe scheme_handler__dispatch__invoke(char *); + probe scheme_handler__dispatch__complete(char *, unsigned int); + probe scheme_handler__return(unsigned int); + probe canon_handler__entry(); + probe canon_handler__dispatch__invoke(char *); + probe canon_handler__dispatch__complete(char *, unsigned int); + probe canon_handler__return(unsigned int); + probe pre_request__entry(); + probe pre_request__dispatch__invoke(char *); + probe pre_request__dispatch__complete(char *, unsigned int); + probe pre_request__return(unsigned int); + probe post_request__entry(); + probe post_request__dispatch__invoke(char *); + probe post_request__dispatch__complete(char *, unsigned int); + probe post_request__return(unsigned int); + + /* Implicit, APR hooks for dav */ + probe gather_propsets__entry(); + probe gather_propsets__dispatch__invoke(char *); + probe gather_propsets__dispatch__complete(char *, unsigned int); + probe gather_propsets__return(unsigned int); + probe find_liveprop__entry(); + probe find_liveprop__dispatch__invoke(char *); + probe find_liveprop__dispatch__complete(char *, unsigned int); + probe find_liveprop__return(unsigned int); + probe insert_all_liveprops__entry(); + probe insert_all_liveprops__dispatch__invoke(char *); + probe insert_all_liveprops__dispatch__complete(char *, unsigned int); + probe insert_all_liveprops__return(unsigned int); +}; Index: include/ap_config.h =================================================================== --- include/ap_config.h (revision 653681) +++ include/ap_config.h (working copy) @@ -238,6 +238,13 @@ #define AP_NONBLOCK_WHEN_MULTI_LISTEN 1 #endif +#if AP_ENABLE_DTRACE && HAVE_SYS_SDT_H +#include +#else +#undef _DTRACE_VERSION +#endif +#include "apache_probes.h" + /* TODO - We need to put OS detection back to make all the following work */ #if defined(SUNOS4) || defined(IRIX) || defined(NEXT) || defined(AUX3) \ Index: configure.in =================================================================== --- configure.in (revision 653681) +++ configure.in (working copy) @@ -381,7 +381,8 @@ strings.h \ sys/prctl.h \ sys/processor.h \ -sys/sem.h +sys/sem.h \ +sys/sdt.h ) AC_HEADER_SYS_WAIT @@ -447,6 +448,25 @@ [if test "$withval" = "yes"; then AC_MSG_ERROR('option --with-sslport requires a value (the SSL TCP port number)'); else SSLPORT="$withval"; fi], [SSLPORT=443]) +AC_ARG_ENABLE(dtrace,APACHE_HELP_STRING(--enable-dtrace,Enable DTrace probes), +[ + enable_dtrace=$enableval + APR_ADDTO(CPPFLAGS, -DAPR_DTRACE_PROVIDER) +], +[ + enable_dtrace=no +]) + +DTRACE=true +if test $enable_dtrace = "yes" -a "$ac_cv_header_sys_sdt_h" = "yes"; then + AC_DEFINE(AP_ENABLE_DTRACE, 1, + [Enable DTrace probes]) + DTRACE="/usr/sbin/dtrace $DTRACEFLAGS" + test -f include/apache_probes.h || $DTRACE -h -s apache_probes.d -o include/apache_probes.h +fi + +APACHE_SUBST(DTRACE) + APR_CHECK_APR_DEFINE(APR_HAVE_IPV6) AC_ARG_ENABLE(v4-mapped,APACHE_HELP_STRING(--enable-v4-mapped,Allow IPv6 sockets to handle IPv4 connections), @@ -504,6 +524,9 @@ AC_SUBST(PILDFLAGS) prefix="$orig_prefix" +cat >static_modules.mk<>static_modules.mk AC_OUTPUT($APACHE_OUTPUT_FILES docs/conf/httpd.conf docs/conf/extra/httpd-autoindex.conf docs/conf/extra/httpd-dav.conf docs/conf/extra/httpd-default.conf docs/conf/extra/httpd-info.conf docs/conf/extra/httpd-languages.conf docs/conf/extra/httpd-manual.conf docs/conf/extra/httpd-mpm.conf docs/conf/extra/httpd-multilang-errordoc.conf docs/conf/extra/httpd-ssl.conf docs/conf/extra/httpd-userdir.conf docs/conf/extra/httpd-vhosts.conf include/ap_config_layout.h support/apxs support/apachectl support/dbmmanage support/envvars-std support/log_server_status support/logresolve.pl support/phf_abuse_log.cgi support/split-logfile build/rules.mk build/pkg/pkginfo build/config_vars.sh,[true],[ APACHE_GEN_MAKEFILES ]) --------------030909060101030802090909--