Return-Path: X-Original-To: archive-asf-public-internal@cust-asf2.ponee.io Delivered-To: archive-asf-public-internal@cust-asf2.ponee.io Received: from cust-asf.ponee.io (cust-asf.ponee.io [163.172.22.183]) by cust-asf2.ponee.io (Postfix) with ESMTP id D3826200B98 for ; Mon, 3 Oct 2016 13:48:06 +0200 (CEST) Received: by cust-asf.ponee.io (Postfix) id D2169160ACC; Mon, 3 Oct 2016 11:48:06 +0000 (UTC) Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by cust-asf.ponee.io (Postfix) with SMTP id 7E1E4160ADC for ; Mon, 3 Oct 2016 13:48:04 +0200 (CEST) Received: (qmail 79063 invoked by uid 500); 3 Oct 2016 11:47:58 -0000 Mailing-List: contact cvs-help@httpd.apache.org; run by ezmlm Precedence: bulk Reply-To: dev@httpd.apache.org list-help: list-unsubscribe: List-Post: List-Id: Delivered-To: mailing list cvs@httpd.apache.org Received: (qmail 79054 invoked by uid 99); 3 Oct 2016 11:47:58 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd4-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 03 Oct 2016 11:47:58 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd4-us-west.apache.org (ASF Mail Server at spamd4-us-west.apache.org) with ESMTP id 125E1C138C for ; Mon, 3 Oct 2016 11:47:58 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd4-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: -1.199 X-Spam-Level: X-Spam-Status: No, score=-1.199 tagged_above=-999 required=6.31 tests=[KAM_ASCII_DIVIDERS=0.8, KAM_LAZY_DOMAIN_SECURITY=1, RP_MATCHES_RCVD=-2.999] autolearn=disabled Received: from mx1-lw-us.apache.org ([10.40.0.8]) by localhost (spamd4-us-west.apache.org [10.40.0.11]) (amavisd-new, port 10024) with ESMTP id AwIEDil63S_4 for ; Mon, 3 Oct 2016 11:47:48 +0000 (UTC) Received: from mailrelay1-us-west.apache.org (mailrelay1-us-west.apache.org [209.188.14.139]) by mx1-lw-us.apache.org (ASF Mail Server at mx1-lw-us.apache.org) with ESMTP id 5CC7D5FB0B for ; Mon, 3 Oct 2016 11:47:48 +0000 (UTC) Received: from svn01-us-west.apache.org (svn.apache.org [10.41.0.6]) by mailrelay1-us-west.apache.org (ASF Mail Server at mailrelay1-us-west.apache.org) with ESMTP id 9D382E0069 for ; Mon, 3 Oct 2016 11:47:47 +0000 (UTC) Received: from svn01-us-west.apache.org (localhost [127.0.0.1]) by svn01-us-west.apache.org (ASF Mail Server at svn01-us-west.apache.org) with ESMTP id 0AE543A027B for ; Mon, 3 Oct 2016 11:47:46 +0000 (UTC) Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Subject: svn commit: r1763158 [1/3] - in /httpd/httpd/trunk: ./ modules/http2/ Date: Mon, 03 Oct 2016 11:47:45 -0000 To: cvs@httpd.apache.org From: icing@apache.org X-Mailer: svnmailer-1.0.9 Message-Id: <20161003114747.0AE543A027B@svn01-us-west.apache.org> archived-at: Mon, 03 Oct 2016 11:48:07 -0000 Author: icing Date: Mon Oct 3 11:47:45 2016 New Revision: 1763158 URL: http://svn.apache.org/viewvc?rev=1763158&view=rev Log: various fixes, mod_cgid interop, response/trailer forwarding rewritten, stability Added: httpd/httpd/trunk/modules/http2/h2_headers.c - copied, changed from r1763154, httpd/httpd/trunk/modules/http2/h2_response.c httpd/httpd/trunk/modules/http2/h2_headers.h - copied, changed from r1763127, httpd/httpd/trunk/modules/http2/h2_response.h Removed: httpd/httpd/trunk/modules/http2/h2_response.c httpd/httpd/trunk/modules/http2/h2_response.h Modified: httpd/httpd/trunk/CHANGES httpd/httpd/trunk/CMakeLists.txt httpd/httpd/trunk/modules/http2/NWGNUmod_http2 httpd/httpd/trunk/modules/http2/config2.m4 httpd/httpd/trunk/modules/http2/h2.h httpd/httpd/trunk/modules/http2/h2_bucket_beam.c httpd/httpd/trunk/modules/http2/h2_bucket_beam.h httpd/httpd/trunk/modules/http2/h2_conn.c httpd/httpd/trunk/modules/http2/h2_conn.h httpd/httpd/trunk/modules/http2/h2_conn_io.c httpd/httpd/trunk/modules/http2/h2_filter.c httpd/httpd/trunk/modules/http2/h2_filter.h httpd/httpd/trunk/modules/http2/h2_from_h1.c httpd/httpd/trunk/modules/http2/h2_from_h1.h httpd/httpd/trunk/modules/http2/h2_h2.c httpd/httpd/trunk/modules/http2/h2_mplx.c httpd/httpd/trunk/modules/http2/h2_mplx.h httpd/httpd/trunk/modules/http2/h2_ngn_shed.c httpd/httpd/trunk/modules/http2/h2_ngn_shed.h httpd/httpd/trunk/modules/http2/h2_proxy_util.c httpd/httpd/trunk/modules/http2/h2_push.c httpd/httpd/trunk/modules/http2/h2_push.h httpd/httpd/trunk/modules/http2/h2_request.c httpd/httpd/trunk/modules/http2/h2_request.h httpd/httpd/trunk/modules/http2/h2_session.c httpd/httpd/trunk/modules/http2/h2_session.h httpd/httpd/trunk/modules/http2/h2_stream.c httpd/httpd/trunk/modules/http2/h2_stream.h httpd/httpd/trunk/modules/http2/h2_task.c httpd/httpd/trunk/modules/http2/h2_task.h httpd/httpd/trunk/modules/http2/h2_util.c httpd/httpd/trunk/modules/http2/h2_util.h httpd/httpd/trunk/modules/http2/h2_version.h httpd/httpd/trunk/modules/http2/mod_http2.c httpd/httpd/trunk/modules/http2/mod_http2.dsp Modified: httpd/httpd/trunk/CHANGES URL: http://svn.apache.org/viewvc/httpd/httpd/trunk/CHANGES?rev=1763158&r1=1763157&r2=1763158&view=diff ============================================================================== --- httpd/httpd/trunk/CHANGES [utf-8] (original) +++ httpd/httpd/trunk/CHANGES [utf-8] Mon Oct 3 11:47:45 2016 @@ -1,6 +1,14 @@ -*- coding: utf-8 -*- Changes with Apache 2.5.0 + *) mod_http2: rewrite of how responses and trailers are transferred between + master and slave connection. Reduction of internal states for tasks + and streams, stability. Heuristic id generation for slave connections + to better keep promise of connection ids unique at given point int time. + Fix for mod_cgid interop in high load situtations. + Fix for handling of incoming trailers when no request body is sent. + [Stefan Eissing] + *) event: Avoid listener periodic wake ups by using the pollset wake-ability when available. PR 57399. [Yann Ylavic, Luca Toscano] Modified: httpd/httpd/trunk/CMakeLists.txt URL: http://svn.apache.org/viewvc/httpd/httpd/trunk/CMakeLists.txt?rev=1763158&r1=1763157&r2=1763158&view=diff ============================================================================== --- httpd/httpd/trunk/CMakeLists.txt (original) +++ httpd/httpd/trunk/CMakeLists.txt Mon Oct 3 11:47:45 2016 @@ -434,7 +434,7 @@ SET(mod_http2_extra_sources modules/http2/h2_from_h1.c modules/http2/h2_h2.c modules/http2/h2_bucket_beam.c modules/http2/h2_mplx.c modules/http2/h2_push.c - modules/http2/h2_request.c modules/http2/h2_response.c + modules/http2/h2_request.c modules/http2/h2_headers.c modules/http2/h2_session.c modules/http2/h2_stream.c modules/http2/h2_switch.c modules/http2/h2_ngn_shed.c modules/http2/h2_task.c modules/http2/h2_util.c Modified: httpd/httpd/trunk/modules/http2/NWGNUmod_http2 URL: http://svn.apache.org/viewvc/httpd/httpd/trunk/modules/http2/NWGNUmod_http2?rev=1763158&r1=1763157&r2=1763158&view=diff ============================================================================== --- httpd/httpd/trunk/modules/http2/NWGNUmod_http2 (original) +++ httpd/httpd/trunk/modules/http2/NWGNUmod_http2 Mon Oct 3 11:47:45 2016 @@ -199,7 +199,7 @@ FILES_nlm_objs = \ $(OBJDIR)/h2_ngn_shed.o \ $(OBJDIR)/h2_push.o \ $(OBJDIR)/h2_request.o \ - $(OBJDIR)/h2_response.o \ + $(OBJDIR)/h2_headers.o \ $(OBJDIR)/h2_session.o \ $(OBJDIR)/h2_stream.o \ $(OBJDIR)/h2_switch.o \ Modified: httpd/httpd/trunk/modules/http2/config2.m4 URL: http://svn.apache.org/viewvc/httpd/httpd/trunk/modules/http2/config2.m4?rev=1763158&r1=1763157&r2=1763158&view=diff ============================================================================== --- httpd/httpd/trunk/modules/http2/config2.m4 (original) +++ httpd/httpd/trunk/modules/http2/config2.m4 Mon Oct 3 11:47:45 2016 @@ -30,11 +30,11 @@ h2_ctx.lo dnl h2_filter.lo dnl h2_from_h1.lo dnl h2_h2.lo dnl +h2_headers.lo dnl h2_mplx.lo dnl h2_ngn_shed.lo dnl h2_push.lo dnl h2_request.lo dnl -h2_response.lo dnl h2_session.lo dnl h2_stream.lo dnl h2_switch.lo dnl Modified: httpd/httpd/trunk/modules/http2/h2.h URL: http://svn.apache.org/viewvc/httpd/httpd/trunk/modules/http2/h2.h?rev=1763158&r1=1763157&r2=1763158&view=diff ============================================================================== --- httpd/httpd/trunk/modules/http2/h2.h (original) +++ httpd/httpd/trunk/modules/http2/h2.h Mon Oct 3 11:47:45 2016 @@ -47,6 +47,9 @@ extern const char *H2_MAGIC_TOKEN; #define H2_HEADER_PATH_LEN 5 #define H2_CRLF "\r\n" +/* Max data size to write so it fits inside a TLS record */ +#define H2_DATA_CHUNK_SIZE ((16*1024) - 100 - 9) + /* Maximum number of padding bytes in a frame, rfc7540 */ #define H2_MAX_PADLEN 256 /* Initial default window size, RFC 7540 ch. 6.5.2 */ @@ -115,38 +118,25 @@ typedef struct h2_session_props { typedef struct h2_request h2_request; struct h2_request { - apr_uint32_t id; /* stream id */ - apr_uint32_t initiated_on; /* initiating stream id (PUSH) or 0 */ - const char *method; /* pseudo header values, see ch. 8.1.2.3 */ const char *scheme; const char *authority; const char *path; apr_table_t *headers; - apr_table_t *trailers; apr_time_t request_time; - apr_off_t content_length; - unsigned int chunked : 1; /* iff requst body needs to be forwarded as chunked */ - unsigned int body : 1; /* iff this request has a body */ + unsigned int chunked : 1; /* iff requst body needs to be forwarded as chunked */ unsigned int serialize : 1; /* iff this request is written in HTTP/1.1 serialization */ - unsigned int push_policy; /* which push policy to use for this request */ }; -typedef struct h2_response h2_response; +typedef struct h2_headers h2_headers; -struct h2_response { - int stream_id; - int rst_error; - int http_status; - apr_off_t content_length; +struct h2_headers { + int status; apr_table_t *headers; - apr_table_t *trailers; - struct h2_response *next; - - const char *sos_filter; + apr_table_t *notes; }; typedef apr_status_t h2_io_data_cb(void *ctx, const char *data, apr_off_t len); @@ -155,7 +145,7 @@ typedef int h2_stream_pri_cmp(int stream /* Note key to attach connection task id to conn_rec/request_rec instances */ -#define H2_TASK_ID_NOTE "http2-task-id" - +#define H2_TASK_ID_NOTE "http2-task-id" +#define H2_FILTER_DEBUG_NOTE "http2-debug" #endif /* defined(__mod_h2__h2__) */ Modified: httpd/httpd/trunk/modules/http2/h2_bucket_beam.c URL: http://svn.apache.org/viewvc/httpd/httpd/trunk/modules/http2/h2_bucket_beam.c?rev=1763158&r1=1763157&r2=1763158&view=diff ============================================================================== --- httpd/httpd/trunk/modules/http2/h2_bucket_beam.c (original) +++ httpd/httpd/trunk/modules/http2/h2_bucket_beam.c Mon Oct 3 11:47:45 2016 @@ -21,6 +21,7 @@ #include #include +#include #include #include "h2_private.h" @@ -170,6 +171,14 @@ const apr_bucket_type_t h2_bucket_type_b * h2_blist, a brigade without allocations ******************************************************************************/ +APR_HOOK_STRUCT( + APR_HOOK_LINK(beam_bucket) +) +AP_IMPLEMENT_HOOK_RUN_FIRST(apr_bucket *, beam_bucket, + (h2_bucket_beam *beam, apr_bucket_brigade *dest, + const apr_bucket *src), + (beam, dest, src), NULL) + apr_size_t h2_util_bl_print(char *buffer, apr_size_t bmax, const char *tag, const char *sep, h2_blist *bl) @@ -518,10 +527,12 @@ void h2_beam_abort(h2_bucket_beam *beam) h2_beam_lock bl; if (enter_yellow(beam, &bl) == APR_SUCCESS) { - r_purge_reds(beam); - h2_blist_cleanup(&beam->red); - beam->aborted = 1; - report_consumption(beam, 0); + if (!beam->aborted) { + beam->aborted = 1; + r_purge_reds(beam); + h2_blist_cleanup(&beam->red); + report_consumption(beam, 0); + } if (beam->m_cond) { apr_thread_cond_broadcast(beam->m_cond); } @@ -792,8 +803,10 @@ transfer: else if (APR_BUCKET_IS_FLUSH(bred)) { bgreen = apr_bucket_flush_create(bb->bucket_alloc); } - else { - /* put red into hold, no green sent out */ + else if (AP_BUCKET_IS_ERROR(bred)) { + ap_bucket_error *eb = (ap_bucket_error *)bred; + bgreen = ap_bucket_error_create(eb->status, eb->data, + bb->p, bb->bucket_alloc); } } else if (APR_BUCKET_IS_FILE(bred)) { @@ -846,6 +859,14 @@ transfer: remain -= bgreen->length; ++transferred; } + else { + bgreen = ap_run_beam_bucket(beam, bb, bred); + while (bgreen && bgreen != APR_BRIGADE_SENTINEL(bb)) { + ++transferred; + remain -= bgreen->length; + bgreen = APR_BUCKET_NEXT(bgreen); + } + } } if (readbytes > 0 && remain < 0) { Modified: httpd/httpd/trunk/modules/http2/h2_bucket_beam.h URL: http://svn.apache.org/viewvc/httpd/httpd/trunk/modules/http2/h2_bucket_beam.h?rev=1763158&r1=1763157&r2=1763158&view=diff ============================================================================== --- httpd/httpd/trunk/modules/http2/h2_bucket_beam.h (original) +++ httpd/httpd/trunk/modules/http2/h2_bucket_beam.h Mon Oct 3 11:47:45 2016 @@ -373,4 +373,8 @@ int h2_beam_was_received(h2_bucket_beam apr_size_t h2_beam_get_files_beamed(h2_bucket_beam *beam); +AP_DECLARE_HOOK(apr_bucket *, beam_bucket, + (h2_bucket_beam *beam, apr_bucket_brigade *dest, + const apr_bucket *src)) + #endif /* h2_bucket_beam_h */ Modified: httpd/httpd/trunk/modules/http2/h2_conn.c URL: http://svn.apache.org/viewvc/httpd/httpd/trunk/modules/http2/h2_conn.c?rev=1763158&r1=1763157&r2=1763158&view=diff ============================================================================== --- httpd/httpd/trunk/modules/http2/h2_conn.c (original) +++ httpd/httpd/trunk/modules/http2/h2_conn.c Mon Oct 3 11:47:45 2016 @@ -14,6 +14,7 @@ */ #include +#include #include @@ -240,12 +241,13 @@ apr_status_t h2_conn_pre_close(struct h2 return status; } -conn_rec *h2_slave_create(conn_rec *master, apr_pool_t *parent, - apr_allocator_t *allocator) +conn_rec *h2_slave_create(conn_rec *master, apr_uint32_t slave_id, + apr_pool_t *parent, apr_allocator_t *allocator) { apr_pool_t *pool; conn_rec *c; void *cfg; + unsigned long l; AP_DEBUG_ASSERT(master); ap_log_cerror(APLOG_MARK, APLOG_TRACE3, 0, master, @@ -271,8 +273,29 @@ conn_rec *h2_slave_create(conn_rec *mast } memcpy(c, master, sizeof(conn_rec)); - - /* Replace these */ + + /* Each conn_rec->id is supposed to be unique at a point in time. Since + * some modules (and maybe external code) uses this id as an identifier + * for the request_rec they handle, it needs to be unique for slave + * connections also. + * The connection id is generated by the MPM and most MPMs use the formula + * id := (child_num * max_threads) + thread_num + * which means that there is a maximum id of about + * idmax := max_child_count * max_threads + * If we assume 2024 child processes with 2048 threads max, we get + * idmax ~= 2024 * 2048 = 2 ** 22 + * On 32 bit systems, we have not much space left, but on 64 bit systems + * (and higher?) we can use the upper 32 bits without fear of collision. + * 32 bits is just what we need, since a connection can only handle so + * many streams. + */ + l = master->id; + if (sizeof(long) >= 8 && l < APR_UINT32_MAX) { + c->id = l|(((unsigned long)slave_id) << 32); + } + else { + c->id = l^(~slave_id); + } c->master = master; c->pool = pool; c->conn_config = ap_create_conn_config(pool); @@ -284,7 +307,8 @@ conn_rec *h2_slave_create(conn_rec *mast c->data_in_output_filters = 0; c->clogging_input_filters = 1; c->log = NULL; - c->log_id = NULL; + c->log_id = apr_psprintf(pool, "%ld-%d", + master->id, slave_id); /* Simulate that we had already a request on this connection. */ c->keepalives = 1; /* We cannot install the master connection socket on the slaves, as @@ -304,6 +328,9 @@ conn_rec *h2_slave_create(conn_rec *mast ap_set_module_config(c->conn_config, h2_conn_mpm_module(), cfg); } + ap_log_cerror(APLOG_MARK, APLOG_TRACE2, 0, c, + "h2_task: creating conn, master=%ld, sid=%ld, logid=%s", + master->id, c->id, c->log_id); return c; } Modified: httpd/httpd/trunk/modules/http2/h2_conn.h URL: http://svn.apache.org/viewvc/httpd/httpd/trunk/modules/http2/h2_conn.h?rev=1763158&r1=1763157&r2=1763158&view=diff ============================================================================== --- httpd/httpd/trunk/modules/http2/h2_conn.h (original) +++ httpd/httpd/trunk/modules/http2/h2_conn.h Mon Oct 3 11:47:45 2016 @@ -66,8 +66,8 @@ typedef enum { h2_mpm_type_t h2_conn_mpm_type(void); -conn_rec *h2_slave_create(conn_rec *master, apr_pool_t *parent, - apr_allocator_t *allocator); +conn_rec *h2_slave_create(conn_rec *master, apr_uint32_t slave_id, + apr_pool_t *parent, apr_allocator_t *allocator); void h2_slave_destroy(conn_rec *slave, apr_allocator_t **pallocator); apr_status_t h2_slave_run_pre_connection(conn_rec *slave, apr_socket_t *csd); Modified: httpd/httpd/trunk/modules/http2/h2_conn_io.c URL: http://svn.apache.org/viewvc/httpd/httpd/trunk/modules/http2/h2_conn_io.c?rev=1763158&r1=1763157&r2=1763158&view=diff ============================================================================== --- httpd/httpd/trunk/modules/http2/h2_conn_io.c (original) +++ httpd/httpd/trunk/modules/http2/h2_conn_io.c Mon Oct 3 11:47:45 2016 @@ -120,8 +120,8 @@ static void h2_conn_io_bb_log(conn_rec * line = *buffer? buffer : "(empty)"; } /* Intentional no APLOGNO */ - ap_log_cerror(APLOG_MARK, level, 0, c, "bb_dump(%ld-%d)-%s: %s", - c->id, stream_id, tag, line); + ap_log_cerror(APLOG_MARK, level, 0, c, "bb_dump(%s)-%s: %s", + c->log_id, tag, line); } Modified: httpd/httpd/trunk/modules/http2/h2_filter.c URL: http://svn.apache.org/viewvc/httpd/httpd/trunk/modules/http2/h2_filter.c?rev=1763158&r1=1763157&r2=1763158&view=diff ============================================================================== --- httpd/httpd/trunk/modules/http2/h2_filter.c (original) +++ httpd/httpd/trunk/modules/http2/h2_filter.c Mon Oct 3 11:47:45 2016 @@ -18,6 +18,7 @@ #include #include #include +#include #include #include #include @@ -32,7 +33,7 @@ #include "h2_task.h" #include "h2_stream.h" #include "h2_request.h" -#include "h2_response.h" +#include "h2_headers.h" #include "h2_stream.h" #include "h2_session.h" #include "h2_util.h" @@ -174,30 +175,92 @@ apr_status_t h2_filter_core_input(ap_fil * http2 connection status handler + stream out source ******************************************************************************/ -static const char *H2_SOS_H2_STATUS = "http2-status"; +typedef struct { + apr_bucket_refcount refcount; + h2_bucket_event_cb *cb; + void *ctx; +} h2_bucket_observer; + +static apr_status_t bucket_read(apr_bucket *b, const char **str, + apr_size_t *len, apr_read_type_e block) +{ + (void)b; + (void)block; + *str = NULL; + *len = 0; + return APR_SUCCESS; +} -int h2_filter_h2_status_handler(request_rec *r) +static void bucket_destroy(void *data) { - h2_ctx *ctx = h2_ctx_rget(r); - h2_task *task; - - if (strcmp(r->handler, "http2-status")) { - return DECLINED; - } - if (r->method_number != M_GET) { - return DECLINED; + h2_bucket_observer *h = data; + if (apr_bucket_shared_destroy(h)) { + if (h->cb) { + h->cb(h->ctx, H2_BUCKET_EV_BEFORE_DESTROY, NULL); + } + apr_bucket_free(h); } +} - task = ctx? h2_ctx_get_task(ctx) : NULL; - if (task) { - /* We need to handle the actual output on the main thread, as - * we need to access h2_session information. */ - apr_table_setn(r->notes, H2_RESP_SOS_NOTE, H2_SOS_H2_STATUS); - apr_table_setn(r->headers_out, "Content-Type", "application/json"); - r->status = 200; - return DONE; +apr_bucket * h2_bucket_observer_make(apr_bucket *b, h2_bucket_event_cb *cb, + void *ctx) +{ + h2_bucket_observer *br; + + br = apr_bucket_alloc(sizeof(*br), b->list); + br->cb = cb; + br->ctx = ctx; + + b = apr_bucket_shared_make(b, br, 0, 0); + b->type = &h2_bucket_type_observer; + return b; +} + +apr_bucket * h2_bucket_observer_create(apr_bucket_alloc_t *list, + h2_bucket_event_cb *cb, void *ctx) +{ + apr_bucket *b = apr_bucket_alloc(sizeof(*b), list); + + APR_BUCKET_INIT(b); + b->free = apr_bucket_free; + b->list = list; + b = h2_bucket_observer_make(b, cb, ctx); + return b; +} + +apr_status_t h2_bucket_observer_fire(apr_bucket *b, h2_bucket_event event) +{ + if (H2_BUCKET_IS_OBSERVER(b)) { + h2_bucket_observer *l = (h2_bucket_observer *)b->data; + return l->cb(l->ctx, event, b); + } + return APR_EINVAL; +} + +const apr_bucket_type_t h2_bucket_type_observer = { + "H2LAZY", 5, APR_BUCKET_METADATA, + bucket_destroy, + bucket_read, + apr_bucket_setaside_noop, + apr_bucket_split_notimpl, + apr_bucket_shared_copy +}; + +apr_bucket *h2_bucket_observer_beam(struct h2_bucket_beam *beam, + apr_bucket_brigade *dest, + const apr_bucket *src) +{ + if (H2_BUCKET_IS_OBSERVER(src)) { + h2_bucket_observer *l = (h2_bucket_observer *)src->data; + apr_bucket *b = h2_bucket_observer_create(dest->bucket_alloc, + l->cb, l->ctx); + APR_BRIGADE_INSERT_TAIL(dest, b); + l->cb = NULL; + l->ctx = NULL; + h2_bucket_observer_fire(b, H2_BUCKET_EV_BEFORE_MASTER_SEND); + return b; } - return DECLINED; + return NULL; } static apr_status_t bbout(apr_bucket_brigade *bb, const char *fmt, ...) @@ -337,31 +400,28 @@ static void add_stats(apr_bucket_brigade bbout(bb, " }%s\n", last? "" : ","); } -static apr_status_t h2_status_stream_filter(h2_stream *stream) +static apr_status_t h2_status_insert(h2_task *task, apr_bucket *b) { - h2_session *s = stream->session; - conn_rec *c = s->c; + h2_mplx *m = task->mplx; + h2_stream *stream = h2_mplx_stream_get(m, task->stream_id); + h2_session *s; + conn_rec *c; + apr_bucket_brigade *bb; + apr_bucket *e; int32_t connFlowIn, connFlowOut; - if (!stream->response) { - return APR_EINVAL; - } - - if (!stream->buffer) { - stream->buffer = apr_brigade_create(stream->pool, c->bucket_alloc); + if (!stream) { + /* stream already done */ + return APR_SUCCESS; } - bb = stream->buffer; + s = stream->session; + c = s->c; - apr_table_unset(stream->response->headers, "Content-Length"); - stream->response->content_length = -1; + bb = apr_brigade_create(stream->pool, c->bucket_alloc); connFlowIn = nghttp2_session_get_effective_local_window_size(s->ngh2); connFlowOut = nghttp2_session_get_remote_window_size(s->ngh2); - apr_table_setn(stream->response->headers, "conn-flow-in", - apr_itoa(stream->pool, connFlowIn)); - apr_table_setn(stream->response->headers, "conn-flow-out", - apr_itoa(stream->pool, connFlowOut)); bbout(bb, "{\n"); bbout(bb, " \"version\": \"draft-01\",\n"); @@ -376,15 +436,96 @@ static apr_status_t h2_status_stream_fil add_stats(bb, s, stream, 1); bbout(bb, "}\n"); + while ((e = APR_BRIGADE_FIRST(bb)) != APR_BRIGADE_SENTINEL(bb)) { + APR_BUCKET_REMOVE(e); + APR_BUCKET_INSERT_AFTER(b, e); + b = e; + } + apr_brigade_destroy(bb); + return APR_SUCCESS; } -apr_status_t h2_stream_filter(h2_stream *stream) +static apr_status_t status_event(void *ctx, h2_bucket_event event, + apr_bucket *b) { - const char *fname = stream->response? stream->response->sos_filter : NULL; - if (fname && !strcmp(H2_SOS_H2_STATUS, fname)) { - return h2_status_stream_filter(stream); + h2_task *task = ctx; + + ap_log_cerror(APLOG_MARK, APLOG_TRACE3, 0, task->c->master, + "status_event(%s): %d", task->id, event); + switch (event) { + case H2_BUCKET_EV_BEFORE_MASTER_SEND: + h2_status_insert(task, b); + break; + default: + break; } return APR_SUCCESS; } +int h2_filter_h2_status_handler(request_rec *r) +{ + h2_ctx *ctx = h2_ctx_rget(r); + conn_rec *c = r->connection; + h2_task *task; + apr_bucket_brigade *bb; + apr_bucket *b; + apr_status_t status; + + if (strcmp(r->handler, "http2-status")) { + return DECLINED; + } + if (r->method_number != M_GET && r->method_number != M_POST) { + return DECLINED; + } + + task = ctx? h2_ctx_get_task(ctx) : NULL; + if (task) { + + if ((status = ap_discard_request_body(r)) != OK) { + return status; + } + + /* We need to handle the actual output on the main thread, as + * we need to access h2_session information. */ + r->status = 200; + r->clength = -1; + r->chunked = 1; + apr_table_unset(r->headers_out, "Content-Length"); + ap_set_content_type(r, "application/json"); + apr_table_setn(r->notes, H2_FILTER_DEBUG_NOTE, "on"); + + bb = apr_brigade_create(r->pool, c->bucket_alloc); + b = h2_bucket_observer_create(c->bucket_alloc, status_event, task); + APR_BRIGADE_INSERT_TAIL(bb, b); + b = apr_bucket_eos_create(c->bucket_alloc); + APR_BRIGADE_INSERT_TAIL(bb, b); + + ap_log_rerror(APLOG_MARK, APLOG_TRACE2, 0, r, + "status_handler(%s): checking for incoming trailers", + task->id); + if (r->trailers_in && !apr_is_empty_table(r->trailers_in)) { + ap_log_rerror(APLOG_MARK, APLOG_TRACE2, 0, r, + "status_handler(%s): seeing incoming trailers", + task->id); + apr_table_setn(r->trailers_out, "h2-trailers-in", + apr_itoa(r->pool, 1)); + } + + status = ap_pass_brigade(r->output_filters, bb); + if (status == APR_SUCCESS + || r->status != HTTP_OK + || c->aborted) { + return OK; + } + else { + /* no way to know what type of error occurred */ + ap_log_rerror(APLOG_MARK, APLOG_TRACE1, status, r, + "status_handler(%s): ap_pass_brigade failed", + task->id); + return AP_FILTER_ERROR; + } + } + return DECLINED; +} + Modified: httpd/httpd/trunk/modules/http2/h2_filter.h URL: http://svn.apache.org/viewvc/httpd/httpd/trunk/modules/http2/h2_filter.h?rev=1763158&r1=1763157&r2=1763158&view=diff ============================================================================== --- httpd/httpd/trunk/modules/http2/h2_filter.h (original) +++ httpd/httpd/trunk/modules/http2/h2_filter.h Mon Oct 3 11:47:45 2016 @@ -16,6 +16,8 @@ #ifndef __mod_h2__h2_filter__ #define __mod_h2__h2_filter__ +struct h2_bucket_beam; +struct h2_headers; struct h2_stream; struct h2_session; @@ -43,9 +45,33 @@ apr_status_t h2_filter_core_input(ap_fil apr_read_type_e block, apr_off_t readbytes); -#define H2_RESP_SOS_NOTE "h2-sos-filter" +/******* observer bucket ******************************************************/ + +typedef enum { + H2_BUCKET_EV_BEFORE_DESTROY, + H2_BUCKET_EV_BEFORE_MASTER_SEND +} h2_bucket_event; + +extern const apr_bucket_type_t h2_bucket_type_observer; + +typedef apr_status_t h2_bucket_event_cb(void *ctx, h2_bucket_event event, apr_bucket *b); + +#define H2_BUCKET_IS_OBSERVER(e) (e->type == &h2_bucket_type_observer) + +apr_bucket * h2_bucket_observer_make(apr_bucket *b, h2_bucket_event_cb *cb, + void *ctx); + +apr_bucket * h2_bucket_observer_create(apr_bucket_alloc_t *list, + h2_bucket_event_cb *cb, void *ctx); + +apr_status_t h2_bucket_observer_fire(apr_bucket *b, h2_bucket_event event); + +apr_bucket *h2_bucket_observer_beam(struct h2_bucket_beam *beam, + apr_bucket_brigade *dest, + const apr_bucket *src); + +/******* /.well-known/h2/state handler ****************************************/ -apr_status_t h2_stream_filter(struct h2_stream *stream); int h2_filter_h2_status_handler(request_rec *r); #endif /* __mod_h2__h2_filter__ */ Modified: httpd/httpd/trunk/modules/http2/h2_from_h1.c URL: http://svn.apache.org/viewvc/httpd/httpd/trunk/modules/http2/h2_from_h1.c?rev=1763158&r1=1763157&r2=1763158&view=diff ============================================================================== --- httpd/httpd/trunk/modules/http2/h2_from_h1.c (original) +++ httpd/httpd/trunk/modules/http2/h2_from_h1.c Mon Oct 3 11:47:45 2016 @@ -28,190 +28,12 @@ #include #include "h2_private.h" -#include "h2_response.h" +#include "h2_headers.h" #include "h2_from_h1.h" #include "h2_task.h" #include "h2_util.h" -static void set_state(h2_from_h1 *from_h1, h2_from_h1_state_t state); - -h2_from_h1 *h2_from_h1_create(int stream_id, apr_pool_t *pool) -{ - h2_from_h1 *from_h1 = apr_pcalloc(pool, sizeof(h2_from_h1)); - if (from_h1) { - from_h1->stream_id = stream_id; - from_h1->pool = pool; - from_h1->state = H2_RESP_ST_STATUS_LINE; - from_h1->hlines = apr_array_make(pool, 10, sizeof(char *)); - } - return from_h1; -} - -static void set_state(h2_from_h1 *from_h1, h2_from_h1_state_t state) -{ - if (from_h1->state != state) { - from_h1->state = state; - } -} - -h2_response *h2_from_h1_get_response(h2_from_h1 *from_h1) -{ - return from_h1->response; -} - -static apr_status_t make_h2_headers(h2_from_h1 *from_h1, request_rec *r) -{ - from_h1->response = h2_response_create(from_h1->stream_id, 0, - from_h1->http_status, - from_h1->hlines, - r->notes, - from_h1->pool); - from_h1->content_length = from_h1->response->content_length; - from_h1->chunked = r->chunked; - - ap_log_cerror(APLOG_MARK, APLOG_DEBUG, 0, r->connection, APLOGNO(03197) - "h2_from_h1(%d): converted headers, content-length: %d" - ", chunked=%d", - from_h1->stream_id, (int)from_h1->content_length, - (int)from_h1->chunked); - - set_state(from_h1, ((from_h1->chunked || from_h1->content_length > 0)? - H2_RESP_ST_BODY : H2_RESP_ST_DONE)); - /* We are ready to be sent to the client */ - return APR_SUCCESS; -} - -static apr_status_t parse_header(h2_from_h1 *from_h1, ap_filter_t* f, - char *line) { - (void)f; - - if (line[0] == ' ' || line[0] == '\t') { - char **plast; - /* continuation line from the header before this */ - while (line[0] == ' ' || line[0] == '\t') { - ++line; - } - - plast = apr_array_pop(from_h1->hlines); - if (plast == NULL) { - /* not well formed */ - return APR_EINVAL; - } - APR_ARRAY_PUSH(from_h1->hlines, const char*) = apr_psprintf(from_h1->pool, "%s %s", *plast, line); - } - else { - /* new header line */ - APR_ARRAY_PUSH(from_h1->hlines, const char*) = apr_pstrdup(from_h1->pool, line); - } - return APR_SUCCESS; -} - -static apr_status_t get_line(h2_from_h1 *from_h1, apr_bucket_brigade *bb, - ap_filter_t* f, char *line, apr_size_t len) -{ - apr_status_t status; - if (!from_h1->bb) { - from_h1->bb = apr_brigade_create(from_h1->pool, f->c->bucket_alloc); - } - else { - apr_brigade_cleanup(from_h1->bb); - } - status = apr_brigade_split_line(from_h1->bb, bb, - APR_BLOCK_READ, - HUGE_STRING_LEN); - if (status == APR_SUCCESS) { - --len; - status = apr_brigade_flatten(from_h1->bb, line, &len); - if (status == APR_SUCCESS) { - /* we assume a non-0 containing line and remove - * trailing crlf. */ - line[len] = '\0'; - if (len >= 2 && !strcmp(H2_CRLF, line + len - 2)) { - len -= 2; - line[len] = '\0'; - } - - apr_brigade_cleanup(from_h1->bb); - ap_log_cerror(APLOG_MARK, APLOG_TRACE1, 0, f->c, - "h2_from_h1(%d): read line: %s", - from_h1->stream_id, line); - } - } - return status; -} - -apr_status_t h2_from_h1_read_response(h2_from_h1 *from_h1, ap_filter_t* f, - apr_bucket_brigade* bb) -{ - apr_status_t status = APR_SUCCESS; - char line[HUGE_STRING_LEN]; - - if ((from_h1->state == H2_RESP_ST_BODY) - || (from_h1->state == H2_RESP_ST_DONE)) { - if (from_h1->chunked) { - /* The httpd core HTTP_HEADER filter has or will install the - * "CHUNK" output transcode filter, which appears further down - * the filter chain. We do not want it for HTTP/2. - * Once we successfully deinstalled it, this filter has no - * further function and we remove it. - */ - status = ap_remove_output_filter_byhandle(f->r->output_filters, - "CHUNK"); - if (status == APR_SUCCESS) { - ap_remove_output_filter(f); - } - } - - return ap_pass_brigade(f->next, bb); - } - - ap_log_cerror(APLOG_MARK, APLOG_TRACE1, 0, f->c, - "h2_from_h1(%d): read_response", from_h1->stream_id); - - while (!APR_BRIGADE_EMPTY(bb) && status == APR_SUCCESS) { - - switch (from_h1->state) { - - case H2_RESP_ST_STATUS_LINE: - case H2_RESP_ST_HEADERS: - status = get_line(from_h1, bb, f, line, sizeof(line)); - if (status != APR_SUCCESS) { - return status; - } - if (from_h1->state == H2_RESP_ST_STATUS_LINE) { - /* instead of parsing, just take it directly */ - from_h1->http_status = f->r->status; - from_h1->state = H2_RESP_ST_HEADERS; - } - else if (line[0] == '\0') { - /* end of headers, create the h2_response and - * pass the rest of the brigade down the filter - * chain. - */ - status = make_h2_headers(from_h1, f->r); - if (from_h1->bb) { - apr_brigade_destroy(from_h1->bb); - from_h1->bb = NULL; - } - if (!APR_BRIGADE_EMPTY(bb)) { - return ap_pass_brigade(f->next, bb); - } - } - else { - status = parse_header(from_h1, f, line); - } - break; - - default: - return ap_pass_brigade(f->next, bb); - } - - } - - return status; -} - /* This routine is called by apr_table_do and merges all instances of * the passed field values into a single array that will be further * processed by some later routine. Originally intended to help split @@ -345,7 +167,7 @@ static int copy_header(void *ctx, const return 1; } -static h2_response *create_response(h2_from_h1 *from_h1, request_rec *r) +static h2_headers *create_response(h2_task *task, request_rec *r) { const char *clheader; const char *ctype; @@ -471,115 +293,316 @@ static h2_response *create_response(h2_f (void *) headers, r->headers_out, NULL); } - return h2_response_rcreate(from_h1->stream_id, r, r->status, - headers, r->pool); + return h2_headers_rcreate(r, r->status, headers, r->pool); } -apr_status_t h2_response_output_filter(ap_filter_t *f, apr_bucket_brigade *bb) +apr_status_t h2_headers_output_filter(ap_filter_t *f, apr_bucket_brigade *bb) { h2_task *task = f->ctx; - h2_from_h1 *from_h1 = task->output.from_h1; request_rec *r = f->r; - apr_bucket *b; + apr_bucket *b, *bresp, *body_bucket = NULL, *next; ap_bucket_error *eb = NULL; + h2_headers *response = NULL; - AP_DEBUG_ASSERT(from_h1 != NULL); - ap_log_cerror(APLOG_MARK, APLOG_TRACE1, 0, f->c, - "h2_from_h1(%d): output_filter called", from_h1->stream_id); + "h2_task(%s): output_filter called", task->id); - if (r->header_only && from_h1->response) { - /* throw away any data after we have compiled the response */ - apr_brigade_cleanup(bb); - return OK; - } - - for (b = APR_BRIGADE_FIRST(bb); - b != APR_BRIGADE_SENTINEL(bb); - b = APR_BUCKET_NEXT(b)) - { - if (AP_BUCKET_IS_ERROR(b) && !eb) { - eb = b->data; - continue; - } - /* - * If we see an EOC bucket it is a signal that we should get out - * of the way doing nothing. - */ - if (AP_BUCKET_IS_EOC(b)) { - ap_remove_output_filter(f); - ap_log_cerror(APLOG_MARK, APLOG_TRACE2, 0, f->c, - "h2_from_h1(%d): eoc bucket passed", - from_h1->stream_id); - return ap_pass_brigade(f->next, bb); + if (!task->output.sent_response) { + /* check, if we need to send the response now. Until we actually + * see a DATA bucket or some EOS/EOR, we do not do so. */ + for (b = APR_BRIGADE_FIRST(bb); + b != APR_BRIGADE_SENTINEL(bb); + b = APR_BUCKET_NEXT(b)) + { + if (AP_BUCKET_IS_ERROR(b) && !eb) { + eb = b->data; + } + else if (AP_BUCKET_IS_EOC(b)) { + /* If we see an EOC bucket it is a signal that we should get out + * of the way doing nothing. + */ + ap_remove_output_filter(f); + ap_log_cerror(APLOG_MARK, APLOG_TRACE2, 0, f->c, + "h2_task(%s): eoc bucket passed", task->id); + return ap_pass_brigade(f->next, bb); + } + else if (!H2_BUCKET_IS_HEADERS(b) && !APR_BUCKET_IS_FLUSH(b)) { + body_bucket = b; + break; + } + } + + if (eb) { + int st = eb->status; + ap_log_cerror(APLOG_MARK, APLOG_DEBUG, 0, f->c, APLOGNO(03047) + "h2_task(%s): err bucket status=%d", task->id, st); + /* throw everything away and replace it with the error response + * generated by ap_die() */ + apr_brigade_cleanup(bb); + ap_die(st, r); + return AP_FILTER_ERROR; + } + + if (body_bucket) { + /* time to insert the response bucket before the body */ + response = create_response(task, r); + if (response == NULL) { + ap_log_cerror(APLOG_MARK, APLOG_NOTICE, 0, f->c, APLOGNO(03048) + "h2_task(%s): unable to create response", task->id); + return APR_ENOMEM; + } + + bresp = h2_bucket_headers_create(f->c->bucket_alloc, response); + APR_BUCKET_INSERT_BEFORE(body_bucket, bresp); + /*APR_BRIGADE_INSERT_HEAD(bb, bresp);*/ + task->output.sent_response = 1; + r->sent_bodyct = 1; } - } - - if (eb) { - int st = eb->status; - apr_brigade_cleanup(bb); - ap_log_cerror(APLOG_MARK, APLOG_DEBUG, 0, f->c, APLOGNO(03047) - "h2_from_h1(%d): err bucket status=%d", - from_h1->stream_id, st); - ap_die(st, r); - return AP_FILTER_ERROR; - } - - from_h1->response = create_response(from_h1, r); - if (from_h1->response == NULL) { - ap_log_cerror(APLOG_MARK, APLOG_NOTICE, 0, f->c, APLOGNO(03048) - "h2_from_h1(%d): unable to create response", - from_h1->stream_id); - return APR_ENOMEM; } if (r->header_only) { ap_log_cerror(APLOG_MARK, APLOG_TRACE1, 0, f->c, - "h2_from_h1(%d): header_only, cleanup output brigade", - from_h1->stream_id); - apr_brigade_cleanup(bb); - return OK; + "h2_task(%s): header_only, cleanup output brigade", + task->id); + b = body_bucket? body_bucket : APR_BRIGADE_FIRST(bb); + while (b != APR_BRIGADE_SENTINEL(bb)) { + next = APR_BUCKET_NEXT(b); + if (APR_BUCKET_IS_EOS(b) || AP_BUCKET_IS_EOR(b)) { + break; + } + APR_BUCKET_REMOVE(b); + apr_bucket_destroy(b); + b = next; + } } - - r->sent_bodyct = 1; /* Whatever follows is real body stuff... */ - - ap_remove_output_filter(f); - if (APLOGctrace1(f->c)) { - apr_off_t len = 0; - apr_brigade_length(bb, 0, &len); - ap_log_cerror(APLOG_MARK, APLOG_TRACE1, 0, f->c, - "h2_from_h1(%d): removed header filter, passing brigade " - "len=%ld", from_h1->stream_id, (long)len); + else if (task->output.sent_response) { + /* lets get out of the way, our task is done */ + ap_remove_output_filter(f); } return ap_pass_brigade(f->next, bb); } -apr_status_t h2_response_trailers_filter(ap_filter_t *f, apr_bucket_brigade *bb) +static void make_chunk(h2_task *task, apr_bucket_brigade *bb, + apr_bucket *first, apr_uint64_t chunk_len, + apr_bucket *tail) +{ + /* Surround the buckets [first, tail[ with new buckets carrying the + * HTTP/1.1 chunked encoding format. If tail is NULL, the chunk extends + * to the end of the brigade. */ + char buffer[128]; + apr_bucket *c; + int len; + + len = apr_snprintf(buffer, H2_ALEN(buffer), + "%"APR_UINT64_T_HEX_FMT"\r\n", chunk_len); + c = apr_bucket_heap_create(buffer, len, NULL, bb->bucket_alloc); + APR_BUCKET_INSERT_BEFORE(first, c); + c = apr_bucket_heap_create("\r\n", 2, NULL, bb->bucket_alloc); + if (tail) { + APR_BUCKET_INSERT_BEFORE(tail, c); + } + else { + APR_BRIGADE_INSERT_TAIL(bb, c); + } +} + +static int ser_header(void *ctx, const char *name, const char *value) +{ + apr_bucket_brigade *bb = ctx; + apr_brigade_printf(bb, NULL, NULL, "%s: %s\r\n", name, value); + return 1; +} + +apr_status_t h2_filter_request_in(ap_filter_t* f, + apr_bucket_brigade* bb, + ap_input_mode_t mode, + apr_read_type_e block, + apr_off_t readbytes) { h2_task *task = f->ctx; - h2_from_h1 *from_h1 = task->output.from_h1; request_rec *r = f->r; - apr_bucket *b; + apr_status_t status = APR_SUCCESS; + apr_bucket *b, *next, *first_data = NULL; + apr_off_t bblen = 0; + + if (!task->input.chunked) { + status = ap_get_brigade(f->next, bb, mode, block, readbytes); + /* pipe data through, just take care of trailers */ + for (b = APR_BRIGADE_FIRST(bb); + b != APR_BRIGADE_SENTINEL(bb); b = next) { + next = APR_BUCKET_NEXT(b); + if (H2_BUCKET_IS_HEADERS(b)) { + h2_headers *headers = h2_bucket_headers_get(b); + ap_assert(headers); + ap_log_rerror(APLOG_MARK, APLOG_TRACE2, 0, r, + "h2_task(%s): receiving trailers", task->id); + r->trailers_in = apr_table_clone(r->pool, headers->headers); + APR_BUCKET_REMOVE(b); + apr_bucket_destroy(b); + ap_remove_input_filter(f); + break; + } + } + return status; + } + + /* Things are more complicated. The standard HTTP input filter, which + * does a lot what we do not want to duplicate, also cares about chunked + * transfer encoding and trailers. + * We need to simulate chunked encoding for it to be happy. + */ + + if (!task->input.bbchunk) { + task->input.bbchunk = apr_brigade_create(r->pool, f->c->bucket_alloc); + } + if (APR_BRIGADE_EMPTY(task->input.bbchunk)) { + /* get more data from the lower layer filters. Always do this + * in larger pieces, since we handle the read modes ourself. + */ + status = ap_get_brigade(f->next, task->input.bbchunk, + AP_MODE_READBYTES, block, 32*1024); + if (status == APR_EOF) { + if (!task->input.eos) { + status = apr_brigade_puts(bb, NULL, NULL, "0\r\n\r\n"); + task->input.eos = 1; + return APR_SUCCESS; + } + ap_remove_input_filter(f); + return status; + + } + else if (status != APR_SUCCESS) { + return status; + } + + ap_log_rerror(APLOG_MARK, APLOG_TRACE2, 0, r, + "h2_task(%s): trailers_in inspecting brigade", task->id); + for (b = APR_BRIGADE_FIRST(task->input.bbchunk); + b != APR_BRIGADE_SENTINEL(task->input.bbchunk) && !task->input.eos; + b = next) { + next = APR_BUCKET_NEXT(b); + if (APR_BUCKET_IS_METADATA(b)) { + if (first_data) { + make_chunk(task, task->input.bbchunk, first_data, bblen, b); + first_data = NULL; + bblen = 0; + } + + if (H2_BUCKET_IS_HEADERS(b)) { + apr_bucket_brigade *tmp; + h2_headers *headers = h2_bucket_headers_get(b); + + ap_assert(headers); + ap_log_rerror(APLOG_MARK, APLOG_TRACE2, 0, r, + "h2_task(%s): receiving trailers", task->id); + tmp = apr_brigade_split_ex(task->input.bbchunk, b, NULL); + if (!apr_is_empty_table(headers->headers)) { + status = apr_brigade_puts(task->input.bbchunk, NULL, NULL, "0\r\n"); + apr_table_do(ser_header, task->input.bbchunk, headers->headers, NULL); + status = apr_brigade_puts(task->input.bbchunk, NULL, NULL, "\r\n"); + } + else { + status = apr_brigade_puts(task->input.bbchunk, NULL, NULL, "0\r\n\r\n"); + } + APR_BRIGADE_CONCAT(task->input.bbchunk, tmp); + apr_brigade_destroy(tmp); + r->trailers_in = apr_table_clone(r->pool, headers->headers); + APR_BUCKET_REMOVE(b); + apr_bucket_destroy(b); + task->input.eos = 1; + } + else if (APR_BUCKET_IS_EOS(b)) { + apr_bucket_brigade *tmp = apr_brigade_split_ex(task->input.bbchunk, b, NULL); + status = apr_brigade_puts(task->input.bbchunk, NULL, NULL, "0\r\n\r\n"); + APR_BRIGADE_CONCAT(task->input.bbchunk, tmp); + apr_brigade_destroy(tmp); + task->input.eos = 1; + } + break; + } + else if (b->length == 0) { + APR_BUCKET_REMOVE(b); + apr_bucket_destroy(b); + } + else { + if (!first_data) { + first_data = b; + } + bblen += b->length; + } + } + + if (first_data) { + make_chunk(task, task->input.bbchunk, first_data, bblen, NULL); + } + } + + if (mode == AP_MODE_EXHAUSTIVE) { + /* return all we have */ + APR_BRIGADE_CONCAT(bb, task->input.bbchunk); + } + else if (mode == AP_MODE_READBYTES) { + status = h2_brigade_concat_length(bb, task->input.bbchunk, readbytes); + } + else if (mode == AP_MODE_SPECULATIVE) { + status = h2_brigade_copy_length(bb, task->input.bbchunk, readbytes); + } + else if (mode == AP_MODE_GETLINE) { + /* we are reading a single LF line, e.g. the HTTP headers. + * this has the nasty side effect to split the bucket, even + * though it ends with CRLF and creates a 0 length bucket */ + status = apr_brigade_split_line(bb, task->input.bbchunk, block, + HUGE_STRING_LEN); + if (APLOGctrace1(f->c)) { + char buffer[1024]; + apr_size_t len = sizeof(buffer)-1; + apr_brigade_flatten(bb, buffer, &len); + buffer[len] = 0; + ap_log_cerror(APLOG_MARK, APLOG_TRACE1, status, f->c, + "h2_task(%s): getline: %s", + task->id, buffer); + } + } + else { + /* Hmm, well. There is mode AP_MODE_EATCRLF, but we chose not + * to support it. Seems to work. */ + ap_log_cerror(APLOG_MARK, APLOG_ERR, APR_ENOTIMPL, f->c, + APLOGNO(02942) + "h2_task, unsupported READ mode %d", mode); + status = APR_ENOTIMPL; + } + + h2_util_bb_log(f->c, task->stream_id, APLOG_TRACE2, "forwarding input", bb); + return status; +} + +apr_status_t h2_filter_trailers_out(ap_filter_t *f, apr_bucket_brigade *bb) +{ + h2_task *task = f->ctx; + request_rec *r = f->r; + apr_bucket *b, *e; - if (from_h1 && from_h1->response) { - /* Detect the EOR bucket and forward any trailers that may have - * been set to our h2_response. + if (task && r) { + /* Detect the EOS/EOR bucket and forward any trailers that may have + * been set to our h2_headers. */ for (b = APR_BRIGADE_FIRST(bb); b != APR_BRIGADE_SENTINEL(bb); b = APR_BUCKET_NEXT(b)) { - if (AP_BUCKET_IS_EOR(b)) { - /* FIXME: need a better test case than this. - apr_table_setn(r->trailers_out, "X", "1"); */ - if (r->trailers_out && !apr_is_empty_table(r->trailers_out)) { - ap_log_cerror(APLOG_MARK, APLOG_DEBUG, 0, f->c, APLOGNO(03049) - "h2_from_h1(%d): trailers filter, saving trailers", - from_h1->stream_id); - h2_response_set_trailers(from_h1->response, - apr_table_clone(from_h1->pool, - r->trailers_out)); - } + if ((APR_BUCKET_IS_EOS(b) || AP_BUCKET_IS_EOR(b)) + && r->trailers_out && !apr_is_empty_table(r->trailers_out)) { + h2_headers *headers; + apr_table_t *trailers; + + ap_log_cerror(APLOG_MARK, APLOG_DEBUG, 0, f->c, APLOGNO(03049) + "h2_task(%s): sending trailers", task->id); + trailers = apr_table_clone(r->pool, r->trailers_out); + headers = h2_headers_rcreate(r, HTTP_OK, trailers, r->pool); + e = h2_bucket_headers_create(bb->bucket_alloc, headers); + APR_BUCKET_INSERT_BEFORE(b, e); + apr_table_clear(r->trailers_out); + ap_remove_output_filter(f); break; } } Modified: httpd/httpd/trunk/modules/http2/h2_from_h1.h URL: http://svn.apache.org/viewvc/httpd/httpd/trunk/modules/http2/h2_from_h1.h?rev=1763158&r1=1763157&r2=1763158&view=diff ============================================================================== --- httpd/httpd/trunk/modules/http2/h2_from_h1.h (original) +++ httpd/httpd/trunk/modules/http2/h2_from_h1.h Mon Oct 3 11:47:45 2016 @@ -30,44 +30,18 @@ * we need to have all handlers and filters involved in request/response * processing, so this seems to be the way for now. */ +struct h2_headers; +struct h2_task; -typedef enum { - H2_RESP_ST_STATUS_LINE, /* parsing http/1 status line */ - H2_RESP_ST_HEADERS, /* parsing http/1 response headers */ - H2_RESP_ST_BODY, /* transferring response body */ - H2_RESP_ST_DONE /* complete response converted */ -} h2_from_h1_state_t; +apr_status_t h2_headers_output_filter(ap_filter_t *f, apr_bucket_brigade *bb); -struct h2_response; +apr_status_t h2_filter_request_in(ap_filter_t* f, + apr_bucket_brigade* brigade, + ap_input_mode_t mode, + apr_read_type_e block, + apr_off_t readbytes); -typedef struct h2_from_h1 h2_from_h1; - -struct h2_from_h1 { - int stream_id; - h2_from_h1_state_t state; - apr_pool_t *pool; - apr_bucket_brigade *bb; - - apr_off_t content_length; - int chunked; - - int http_status; - apr_array_header_t *hlines; - - struct h2_response *response; -}; - - -h2_from_h1 *h2_from_h1_create(int stream_id, apr_pool_t *pool); - -apr_status_t h2_from_h1_read_response(h2_from_h1 *from_h1, - ap_filter_t* f, apr_bucket_brigade* bb); - -struct h2_response *h2_from_h1_get_response(h2_from_h1 *from_h1); - -apr_status_t h2_response_output_filter(ap_filter_t *f, apr_bucket_brigade *bb); - -apr_status_t h2_response_trailers_filter(ap_filter_t *f, apr_bucket_brigade *bb); +apr_status_t h2_filter_trailers_out(ap_filter_t *f, apr_bucket_brigade *bb); void h2_from_h1_set_basic_http_header(apr_table_t *headers, request_rec *r, apr_pool_t *pool); Modified: httpd/httpd/trunk/modules/http2/h2_h2.c URL: http://svn.apache.org/viewvc/httpd/httpd/trunk/modules/http2/h2_h2.c?rev=1763158&r1=1763157&r2=1763158&view=diff ============================================================================== --- httpd/httpd/trunk/modules/http2/h2_h2.c (original) +++ httpd/httpd/trunk/modules/http2/h2_h2.c Mon Oct 3 11:47:45 2016 @@ -32,12 +32,15 @@ #include "mod_http2.h" #include "h2_private.h" +#include "h2_bucket_beam.h" #include "h2_stream.h" #include "h2_task.h" #include "h2_config.h" #include "h2_ctx.h" #include "h2_conn.h" +#include "h2_filter.h" #include "h2_request.h" +#include "h2_headers.h" #include "h2_session.h" #include "h2_util.h" #include "h2_h2.h" @@ -569,6 +572,10 @@ void h2_h2_register_hooks(void) */ ap_hook_post_read_request(h2_h2_post_read_req, NULL, NULL, APR_HOOK_REALLY_FIRST); ap_hook_fixups(h2_h2_late_fixups, NULL, NULL, APR_HOOK_LAST); + + /* special bucket type transfer through a h2_bucket_beam */ + ap_hook_beam_bucket(h2_bucket_observer_beam, NULL, NULL, APR_HOOK_MIDDLE); + ap_hook_beam_bucket(h2_bucket_headers_beam, NULL, NULL, APR_HOOK_MIDDLE); } int h2_h2_process_conn(conn_rec* c) @@ -684,30 +691,23 @@ static int h2_h2_post_read_req(request_r * that we manipulate filters only once. */ if (task && !task->filters_set) { ap_filter_t *f; + ap_log_rerror(APLOG_MARK, APLOG_TRACE3, 0, r, "adding request filters"); - /* setup the correct output filters to process the response - * on the proper mod_http2 way. */ - ap_log_rerror(APLOG_MARK, APLOG_TRACE3, 0, r, "adding task output filter"); - if (task->ser_headers) { - ap_add_output_filter("H1_TO_H2_RESP", task, r, r->connection); - } - else { - /* replace the core http filter that formats response headers - * in HTTP/1 with our own that collects status and headers */ - ap_remove_output_filter_byhandle(r->output_filters, "HTTP_HEADER"); - ap_add_output_filter("H2_RESPONSE", task, r, r->connection); - } + /* setup the correct filters to process the request for h2 */ + ap_add_input_filter("H2_REQUEST", task, r, r->connection); + + /* replace the core http filter that formats response headers + * in HTTP/1 with our own that collects status and headers */ + ap_remove_output_filter_byhandle(r->output_filters, "HTTP_HEADER"); + ap_add_output_filter("H2_RESPONSE", task, r, r->connection); - /* trailers processing. Incoming trailers are added to this - * request via our h2 input filter, outgoing trailers - * in a special h2 out filter. */ for (f = r->input_filters; f; f = f->next) { - if (!strcmp("H2_TO_H1", f->frec->name)) { + if (!strcmp("H2_SLAVE_IN", f->frec->name)) { f->r = r; break; } } - ap_add_output_filter("H2_TRAILERS", task, r, r->connection); + ap_add_output_filter("H2_TRAILERS_OUT", task, r, r->connection); task->filters_set = 1; } } Copied: httpd/httpd/trunk/modules/http2/h2_headers.c (from r1763154, httpd/httpd/trunk/modules/http2/h2_response.c) URL: http://svn.apache.org/viewvc/httpd/httpd/trunk/modules/http2/h2_headers.c?p2=httpd/httpd/trunk/modules/http2/h2_headers.c&p1=httpd/httpd/trunk/modules/http2/h2_response.c&r1=1763154&r2=1763158&rev=1763158&view=diff ============================================================================== --- httpd/httpd/trunk/modules/http2/h2_response.c (original) +++ httpd/httpd/trunk/modules/http2/h2_headers.c Mon Oct 3 11:47:45 2016 @@ -26,194 +26,136 @@ #include #include "h2_private.h" -#include "h2_filter.h" #include "h2_h2.h" #include "h2_util.h" #include "h2_request.h" -#include "h2_response.h" +#include "h2_headers.h" -static apr_table_t *parse_headers(apr_array_header_t *hlines, apr_pool_t *pool) +typedef struct { + apr_bucket_refcount refcount; + h2_headers *headers; +} h2_bucket_headers; + +static apr_status_t bucket_read(apr_bucket *b, const char **str, + apr_size_t *len, apr_read_type_e block) { - if (hlines) { - apr_table_t *headers = apr_table_make(pool, hlines->nelts); - int i; - - for (i = 0; i < hlines->nelts; ++i) { - char *hline = ((char **)hlines->elts)[i]; - char *sep = ap_strchr(hline, ':'); - if (!sep) { - ap_log_perror(APLOG_MARK, APLOG_WARNING, APR_EINVAL, pool, - APLOGNO(02955) "h2_response: invalid header[%d] '%s'", - i, (char*)hline); - /* not valid format, abort */ - return NULL; - } - (*sep++) = '\0'; - while (*sep == ' ' || *sep == '\t') { - ++sep; - } - - if (!h2_util_ignore_header(hline)) { - apr_table_merge(headers, hline, sep); - } - } - return headers; - } - else { - return apr_table_make(pool, 0); - } + (void)b; + (void)block; + *str = NULL; + *len = 0; + return APR_SUCCESS; } -static const char *get_sos_filter(apr_table_t *notes) +apr_bucket * h2_bucket_headers_make(apr_bucket *b, h2_headers *r) { - return notes? apr_table_get(notes, H2_RESP_SOS_NOTE) : NULL; -} + h2_bucket_headers *br; -static void check_clen(h2_response *response, request_rec *r, apr_pool_t *pool) -{ + br = apr_bucket_alloc(sizeof(*br), b->list); + br->headers = r; + + b = apr_bucket_shared_make(b, br, 0, 0); + b->type = &h2_bucket_type_headers; - if (r && r->header_only) { - response->content_length = 0; - } - else if (response->headers) { - const char *s = apr_table_get(response->headers, "Content-Length"); - if (s) { - char *end; - response->content_length = apr_strtoi64(s, &end, 10); - if (s == end) { - ap_log_perror(APLOG_MARK, APLOG_WARNING, APR_EINVAL, - pool, APLOGNO(02956) - "h2_response: content-length" - " value not parsed: %s", s); - response->content_length = -1; - } - } - } -} + return b; +} -static h2_response *h2_response_create_int(int stream_id, - int rst_error, - int http_status, - apr_table_t *headers, - apr_table_t *notes, - apr_pool_t *pool) +apr_bucket * h2_bucket_headers_create(apr_bucket_alloc_t *list, + h2_headers *r) { - h2_response *response; + apr_bucket *b = apr_bucket_alloc(sizeof(*b), list); - if (!headers) { - return NULL; + APR_BUCKET_INIT(b); + b->free = apr_bucket_free; + b->list = list; + b = h2_bucket_headers_make(b, r); + return b; +} + +h2_headers *h2_bucket_headers_get(apr_bucket *b) +{ + if (H2_BUCKET_IS_HEADERS(b)) { + return ((h2_bucket_headers *)b->data)->headers; } - - response = apr_pcalloc(pool, sizeof(h2_response)); - if (response == NULL) { - return NULL; + return NULL; +} + +const apr_bucket_type_t h2_bucket_type_headers = { + "H2HEADERS", 5, APR_BUCKET_METADATA, + apr_bucket_destroy_noop, + bucket_read, + apr_bucket_setaside_noop, + apr_bucket_split_notimpl, + apr_bucket_shared_copy +}; + +apr_bucket *h2_bucket_headers_beam(struct h2_bucket_beam *beam, + apr_bucket_brigade *dest, + const apr_bucket *src) +{ + if (H2_BUCKET_IS_HEADERS(src)) { + h2_headers *r = ((h2_bucket_headers *)src->data)->headers; + apr_bucket *b = h2_bucket_headers_create(dest->bucket_alloc, r); + APR_BRIGADE_INSERT_TAIL(dest, b); + return b; } - - response->stream_id = stream_id; - response->rst_error = rst_error; - response->http_status = http_status? http_status : 500; - response->content_length = -1; - response->headers = headers; - response->sos_filter = get_sos_filter(notes); - - check_clen(response, NULL, pool); - return response; + return NULL; } -h2_response *h2_response_create(int stream_id, - int rst_error, - int http_status, - apr_array_header_t *hlines, - apr_table_t *notes, - apr_pool_t *pool) +h2_headers *h2_headers_create(int status, apr_table_t *headers_in, + apr_table_t *notes, apr_pool_t *pool) { - return h2_response_create_int(stream_id, rst_error, http_status, - parse_headers(hlines, pool), notes, pool); + h2_headers *headers = apr_pcalloc(pool, sizeof(h2_headers)); + headers->status = status; + headers->headers = (headers_in? apr_table_copy(pool, headers_in) + : apr_table_make(pool, 5)); + headers->notes = (notes? apr_table_copy(pool, notes) + : apr_table_make(pool, 5)); + return headers; } -h2_response *h2_response_rcreate(int stream_id, request_rec *r, int status, +h2_headers *h2_headers_rcreate(request_rec *r, int status, apr_table_t *header, apr_pool_t *pool) { - h2_response *response = apr_pcalloc(pool, sizeof(h2_response)); - if (response == NULL) { - return NULL; - } - - response->stream_id = stream_id; - response->http_status = status; - response->content_length = -1; - response->headers = header? header : apr_table_make(pool, 5); - response->sos_filter = get_sos_filter(r->notes); - - check_clen(response, r, pool); - - if (response->http_status == HTTP_FORBIDDEN) { + h2_headers *headers = h2_headers_create(status, header, r->notes, pool); + if (headers->status == HTTP_FORBIDDEN) { const char *cause = apr_table_get(r->notes, "ssl-renegotiate-forbidden"); if (cause) { /* This request triggered a TLS renegotiation that is now allowed * in HTTP/2. Tell the client that it should use HTTP/1.1 for this. */ - ap_log_rerror(APLOG_MARK, APLOG_DEBUG, response->http_status, r, + ap_log_rerror(APLOG_MARK, APLOG_DEBUG, headers->status, r, APLOGNO(03061) - "h2_response(%ld-%d): renegotiate forbidden, cause: %s", - (long)r->connection->id, stream_id, cause); - response->rst_error = H2_ERR_HTTP_1_1_REQUIRED; + "h2_headers(%ld): renegotiate forbidden, cause: %s", + (long)r->connection->id, cause); + headers->status = H2_ERR_HTTP_1_1_REQUIRED; } } - - return response; + return headers; } -h2_response *h2_response_die(int stream_id, apr_status_t type, - const struct h2_request *req, apr_pool_t *pool) +h2_headers *h2_headers_die(apr_status_t type, + const h2_request *req, apr_pool_t *pool) { - apr_table_t *headers = apr_table_make(pool, 5); - char *date = NULL; - int status = (type >= 200 && type < 600)? type : 500; + h2_headers *headers; + char *date; + headers = apr_pcalloc(pool, sizeof(h2_headers)); + headers->status = (type >= 200 && type < 600)? type : 500; + headers->headers = apr_table_make(pool, 5); + headers->notes = apr_table_make(pool, 5); + date = apr_palloc(pool, APR_RFC822_DATE_LEN); ap_recent_rfc822_date(date, req? req->request_time : apr_time_now()); - apr_table_setn(headers, "Date", date); - apr_table_setn(headers, "Server", ap_get_server_banner()); + apr_table_setn(headers->headers, "Date", date); + apr_table_setn(headers->headers, "Server", ap_get_server_banner()); - return h2_response_create_int(stream_id, 0, status, headers, NULL, pool); + return headers; } -h2_response *h2_response_clone(apr_pool_t *pool, h2_response *from) +int h2_headers_are_response(h2_headers *headers) { - h2_response *to = apr_pcalloc(pool, sizeof(h2_response)); - - to->stream_id = from->stream_id; - to->http_status = from->http_status; - to->content_length = from->content_length; - to->sos_filter = from->sos_filter; - if (from->headers) { - to->headers = apr_table_clone(pool, from->headers); - } - if (from->trailers) { - to->trailers = apr_table_clone(pool, from->trailers); - } - return to; + return headers->status >= 200; } -void h2_response_set_trailers(h2_response *response, apr_table_t *trailers) -{ - response->trailers = trailers; -} - -int h2_response_is_final(h2_response *response) -{ - return response->http_status >= 200; -} - -h2_response *h2_response_get_final(h2_response *response) -{ - for (/**/; response; response = response->next) { - if (h2_response_is_final(response)) { - return response; - } - } - return NULL; -} Copied: httpd/httpd/trunk/modules/http2/h2_headers.h (from r1763127, httpd/httpd/trunk/modules/http2/h2_response.h) URL: http://svn.apache.org/viewvc/httpd/httpd/trunk/modules/http2/h2_headers.h?p2=httpd/httpd/trunk/modules/http2/h2_headers.h&p1=httpd/httpd/trunk/modules/http2/h2_response.h&r1=1763127&r2=1763158&rev=1763158&view=diff ============================================================================== --- httpd/httpd/trunk/modules/http2/h2_response.h (original) +++ httpd/httpd/trunk/modules/http2/h2_headers.h Mon Oct 3 11:47:45 2016 @@ -13,64 +13,58 @@ * limitations under the License. */ -#ifndef __mod_h2__h2_response__ -#define __mod_h2__h2_response__ +#ifndef __mod_h2__h2_headers__ +#define __mod_h2__h2_headers__ #include "h2.h" +struct h2_bucket_beam; + +extern const apr_bucket_type_t h2_bucket_type_headers; + +#define H2_BUCKET_IS_HEADERS(e) (e->type == &h2_bucket_type_headers) + +apr_bucket * h2_bucket_headers_make(apr_bucket *b, h2_headers *r); + +apr_bucket * h2_bucket_headers_create(apr_bucket_alloc_t *list, + h2_headers *r); + +h2_headers *h2_bucket_headers_get(apr_bucket *b); + +apr_bucket *h2_bucket_headers_beam(struct h2_bucket_beam *beam, + apr_bucket_brigade *dest, + const apr_bucket *src); + /** - * Create the response from the status and parsed header lines. - * @param stream_id id of the stream to create the response for - * @param rst_error error for reset or 0 - * @param http_status http status code of response - * @param hlines the text lines of the response header + * Create the headers from the given status and headers + * @param status the headers status + * @param header the headers of the headers + * @param notes the notes carried by the headers * @param pool the memory pool to use */ -h2_response *h2_response_create(int stream_id, - int rst_error, - int http_status, - apr_array_header_t *hlines, - apr_table_t *notes, - apr_pool_t *pool); +h2_headers *h2_headers_create(int status, apr_table_t *header, + apr_table_t *notes, apr_pool_t *pool); /** - * Create the response from the given request_rec. - * @param stream_id id of the stream to create the response for + * Create the headers from the given request_rec. * @param r the request record which was processed - * @param header the headers of the response + * @param status the headers status + * @param header the headers of the headers * @param pool the memory pool to use */ -h2_response *h2_response_rcreate(int stream_id, request_rec *r, int status, +h2_headers *h2_headers_rcreate(request_rec *r, int status, apr_table_t *header, apr_pool_t *pool); /** - * Create the response for the given error. - * @param stream_id id of the stream to create the response for + * Create the headers for the given error. + * @param stream_id id of the stream to create the headers for * @param type the error code * @param req the original h2_request * @param pool the memory pool to use */ -h2_response *h2_response_die(int stream_id, apr_status_t type, +h2_headers *h2_headers_die(apr_status_t type, const struct h2_request *req, apr_pool_t *pool); -/** - * Deep copies the response into a new pool. - * @param pool the pool to use for the clone - * @param from the response to clone - * @return the cloned response - */ -h2_response *h2_response_clone(apr_pool_t *pool, h2_response *from); - -/** - * Set the trailers in the response. Will replace any existing trailers. Will - * *not* clone the table. - * - * @param response the repsone to set the trailers for - * @param trailers the trailers to set - */ -void h2_response_set_trailers(h2_response *response, apr_table_t *trailers); - -int h2_response_is_final(h2_response *response); -h2_response *h2_response_get_final(h2_response *response); +int h2_headers_are_response(h2_headers *headers); -#endif /* defined(__mod_h2__h2_response__) */ +#endif /* defined(__mod_h2__h2_headers__) */