Return-Path: Delivered-To: apmail-httpd-cvs-archive@www.apache.org Received: (qmail 3162 invoked from network); 19 Mar 2007 14:40:02 -0000 Received: from hermes.apache.org (HELO mail.apache.org) (140.211.11.2) by minotaur.apache.org with SMTP; 19 Mar 2007 14:40:02 -0000 Received: (qmail 37255 invoked by uid 500); 19 Mar 2007 14:40:10 -0000 Delivered-To: apmail-httpd-cvs-archive@httpd.apache.org Received: (qmail 37211 invoked by uid 500); 19 Mar 2007 14:40:09 -0000 Mailing-List: contact cvs-help@httpd.apache.org; run by ezmlm Precedence: bulk Reply-To: dev@httpd.apache.org list-help: list-unsubscribe: List-Post: List-Id: Delivered-To: mailing list cvs@httpd.apache.org Received: (qmail 37200 invoked by uid 99); 19 Mar 2007 14:40:09 -0000 Received: from herse.apache.org (HELO herse.apache.org) (140.211.11.133) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 19 Mar 2007 07:40:09 -0700 X-ASF-Spam-Status: No, hits=-99.5 required=10.0 tests=ALL_TRUSTED,NO_REAL_NAME X-Spam-Check-By: apache.org Received: from [140.211.11.3] (HELO eris.apache.org) (140.211.11.3) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 19 Mar 2007 07:40:01 -0700 Received: by eris.apache.org (Postfix, from userid 65534) id E0CE41A9838; Mon, 19 Mar 2007 07:39:40 -0700 (PDT) Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Subject: svn commit: r519960 - /httpd/httpd/trunk/docs/manual/developer/output-filters.xml Date: Mon, 19 Mar 2007 14:39:40 -0000 To: cvs@httpd.apache.org From: jorton@apache.org X-Mailer: svnmailer-1.1.0 Message-Id: <20070319143940.E0CE41A9838@eris.apache.org> X-Virus-Checked: Checked by ClamAV on apache.org Author: jorton Date: Mon Mar 19 07:39:39 2007 New Revision: 519960 URL: http://svn.apache.org/viewvc?view=rev&rev=519960 Log: Updates based on feedback from Ruediger and Jeff: - correct use of "up" and "down" w.r.t. filter chain - various grammar fixes - recommend use of apr_brigade_cleanup() in the brigade structure section - fix use of APR_BRIGADE_INSERT_HEAD in an example - expand section on use of ap_save_brigade() - simplify rule regarding fixed processing fixed # of buckets Modified: httpd/httpd/trunk/docs/manual/developer/output-filters.xml Modified: httpd/httpd/trunk/docs/manual/developer/output-filters.xml URL: http://svn.apache.org/viewvc/httpd/httpd/trunk/docs/manual/developer/output-filters.xml?view=diff&rev=519960&r1=519959&r2=519960 ============================================================================== --- httpd/httpd/trunk/docs/manual/developer/output-filters.xml (original) +++ httpd/httpd/trunk/docs/manual/developer/output-filters.xml Mon Mar 19 07:39:39 2007 @@ -52,7 +52,7 @@

A filter can tell whether a bucket represents either data or metadata using the APR_BUCKET_IS_METADATA macro. - Generally, all metadata buckets should be passed up the filter + Generally, all metadata buckets should be passed down the filter chain by an output filter. Filters may transform, delete, and insert data buckets as appropriate.

@@ -65,14 +65,14 @@ applicable) down the filter chain immediately.

FLUSH buckets are sent when the - content generator (or a downstream filter) knows that there may be + content generator (or an upstream filter) knows that there may be a delay before more content can be sent. By passing - FLUSH buckets up the filter chain immediately, + FLUSH buckets down the filter chain immediately, filters ensure that the client is not kept waiting for pending data longer than necessary. -

Filters can create FLUSH buckets and pass these up - the filter chain if desired. Generating FLUSH +

Filters can create FLUSH buckets and pass these + down the filter chain if desired. Generating FLUSH buckets unnecessarily, or too frequently, can harm network utilisation since it may force large numbers of small packets to be sent, rather than a small number of larger packets. The @@ -94,9 +94,9 @@ Filter invocation

For any given request, an output filter might be invoked only - once and be given a single brigade representing the entire response. + once and be given a single brigade representing the entire response. It is also possible that the number of times a filter is invoked - for single response is proportional to the size of the content + for a single response is proportional to the size of the content being filtered, with the filter being passed a brigade containing a single bucket each time. Filters must operate correctly in either case.

@@ -112,7 +112,7 @@ the brigade. Any buckets in the brigade after an EOS should be ignored.

-

An output filter should never pass an empty brigade up the +

An output filter should never pass an empty brigade down the filter chain. But, for good defensive programming, filters should be prepared to accept an empty brigade, and do nothing.

@@ -169,15 +169,18 @@ first invocation per request, and store that brigade in its state structure.

- It is generally never advisable to use +

It is generally never advisable to use apr_brigade_destroy to "destroy" a brigade. The memory used by the brigade structure will not be released by calling this function (since it comes from a pool), but the associated pool cleanup is unregistered. Using apr_brigade_destroy can in fact cause memory leaks; - if a "destroyed" brigade contains still contains buckets when its + if a "destroyed" brigade contains buckets when its containing pool is destroyed, those buckets will not be - immediately destroyed. + immediately destroyed.

+ +

In general, filters should use apr_brigade_cleanup + in preference to apr_brigade_destroy.

 @@ -193,9 +196,9 @@
  • The amount of data represented by the bucket may or may not have a determinate length; for a bucket which represents data of indeterminate length, the ->length field is set to - the value (apr_size_t)-1. The PIPE - bucket type is an example of a bucket type has an indeterminate - length; it represents the output from a pipe, .
    • + the value (apr_size_t)-1. For example, buckets of + the PIPE bucket type have an indeterminate length; + they represent the output from a pipe.
  • The data represented by a bucket may or may not be mapped into memory. The FILE bucket type, for example, @@ -255,7 +258,7 @@ apr_bucket_read call morphed a FILE bucket into a HEAP bucket.

    -

    In contrast, the implementation below will use consume a fixed +

    In contrast, the implementation below will consume a fixed amount of memory to filter any brigade; a temporary brigade is needed and must be allocated only once per response, see the Maintaining state section.

    @@ -272,8 +275,8 @@ /* Remove bucket e from bb. */ APR_BUCKET_REMOVE(e); /* Insert it into temporary brigade. */ - APR_BRIGADE_INSERT_HEAD(tmpbb); - /* Pass brigade upstream. */ + APR_BRIGADE_INSERT_HEAD(tmpbb, e); + /* Pass brigade downstream. */ rv = ap_pass_brigade(f->next, tmpbb); if (rv) ...; apr_brigade_cleanup(tmpbb); @@ -334,17 +337,33 @@ apr_bucket_setaside function.

    Alternatively, the ap_save_brigade function can be - used, which will create a new brigade containing buckets with a - lifetime as long as the given pool argument. This function must - be used with great care, however: on return it guarantees that all - the buckets in the returned brigade will represent data mapped - into memory. If given an input brigade containing, for example, a - PIPE bucket, ap_save_brigade will consume an - arbitrary amount of memory to store the entire output of the - pipe.

    + used, which will move all the buckets into a separate brigade + containing buckets with a lifetime as long as the given pool + argument. This function must be used with care, taking into + account the following points: + +
      +
    1. On return, ap_save_brigade guarantees that all + the buckets in the returned brigade will represent data mapped + into memory. If given an input brigade containing, for example, + a PIPE bucket, ap_save_brigade will + consume an arbitrary amount of memory to store the entire output + of the pipe.
      2. + +
    2. When ap_save_brigade reads from buckets which + cannot be setaside, it will always perform blocking reads, + removing the opportunity to use Non-blocking + bucket reads.
      3. + +
    3. If ap_save_brigade is used without passing a + non-NULL "saveto" (destination) brigade parameter, + the function will create a new brigade, which may cause memory + use to be proportional to content size as described in the Brigade structure section.
      4. +

    Filters must ensure that any buffered data is - processed and passed up the filter chain during the last + processed and passed down the filter chain during the last invocation for a given response (a brigade containing an EOS bucket). Otherwise such data will be lost. @@ -359,9 +378,9 @@ from the data source. A good filter will first attempt to read from every data bucket using a non-blocking read; if that fails with APR_EAGAIN, then send a FLUSH - bucket up the filter chain, and retry using a blocking read.

    + bucket down the filter chain, and retry using a blocking read.

    -

    This mode of operation ensure that any filters further up the +

    This mode of operation ensure that any filters further down the filter chain will flush any buffered buckets if a slow content source is being used.

    @@ -382,7 +401,7 @@ rv = apr_bucket_read(e, &data, &length, mode); if (rv == APR_EAGAIN && mode == APR_NONBLOCK_READ) { - /* Pass up a brigade containing a flush bucket: */ + /* Pass down a brigade containing a flush bucket: */ APR_BRIGADE_INSERT_TAIL(tmpbb, apr_bucket_flush_create(...)); rv = ap_pass_brigade(f->next, tmpbb); apr_brigade_cleanup(tmpbb); @@ -409,29 +428,28 @@ follow:

      -
    1. Output filters should not pass empty brigades up the filter +
    2. Output filters should not pass empty brigades down the filter chain, but should be tolerant of being passed empty brigades.
      3. -
    3. Output filters must pass all metadata buckets up the filter +
    4. Output filters must pass all metadata buckets down the filter chain; FLUSH buckets should be respected by passing - any pending or buffered buckets up the filter chain.
      5. + any pending or buffered buckets down the filter chain.
    5. Output filters should ignore any buckets following an EOS bucket.
      6. -
    6. Output filters which read all the buckets in a brigade must - process a fixed number of buckets (or amount of data) at a time, - to ensure that memory consumption is not proportional to the - size of the content being filtered.
      7. +
    7. Output filters must process a fixed amount of data at a + time, to ensure that memory consumption is not proportional to + the size of the content being filtered.
    8. Output filters should be agnostic with respect to bucket types, and must be able to process buckets of unfamiliar type.
    9. After calling ap_pass_brigade to pass a brigade - up the filter chain, output filters should call - apr_brigade_clear to ensure the brigade is empty + down the filter chain, output filters should call + apr_brigade_cleanup to ensure the brigade is empty before reusing that brigade structure; output filters should never use apr_brigade_destroy to "destroy" brigades.
      10. @@ -441,13 +459,13 @@
    10. Output filters must not ignore the return value of ap_pass_brigade, and must return appropriate errors - back down the filter chain.
      11. + back up the filter chain.
    11. Output filters must only create a fixed number of bucket brigades for each response, rather than one per invocation.
    12. Output filters should first attempt non-blocking reads from - each data bucket, and send a FLUSH bucket up the + each data bucket, and send a FLUSH bucket down the filter chain if the read blocks, before retrying with a blocking read.