httpd-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Greg Stein <>
Subject Re: byterange filter + apr_brigade_partition
Date Fri, 08 Jun 2001 20:58:58 GMT
On Fri, Jun 08, 2001 at 03:10:07PM -0400, Cliff Woolley wrote:
> On Fri, 8 Jun 2001, Greg Stein wrote:
> > > catastrophic happens between the calls to apr_brigade_length() and
> > > apr_brigade_partition().  That's why it worked fine before to just ignore
> > > the possibility of a NULL return from apr_brigade_partition()... it just
> > > shouldn't ever happen.
> >
> > It can definitely happen.
> >
> > Consider a brigade that contains a PIPE bucket from a CGI. When we go to
> > partition it, we read from the PIPE. Are you *sure* that read will never
> > return an error on us? :-)
> But there won't be a PIPE bucket in the input brigade by the time it gets
> to the apr_brigade_partition() calls, because apr_brigade_length() will
> have already seen and read from any such buckets, morphing them to HEAP
> buckets.  All apr_brigade_partition() will ever see is the "plain"
> buckets, ie ones that you don't need to read from to operate on.  If the
> read on the (former) PIPE/SOCKET buckets was going to fail, it would have
> failed earlier in the call to apr_brigade_length().  Right?

Good point. (and let's also clarify to ourselves that we're just talking
about the bytrange filter's use of partition; other partition users will
still need to be wary of error return values)

However... (and you knew that was coming :-)

Not if the PIPE has a known length, but simply hasn't read the data yet
(dunno if PIPE can support this; if not, then just say there is another
bucket that does). Let's say you have some kind of protocol via pipe or a
socket, where the length is known, but we haven't read the data (into a heap
bucket yet). You insert a bucket with length == <whatever>. Thus, the
length() call works without a read() call.

Now the partition comes along and figures out that a split needs to occur
right in the middle of this PIPE' bucket. We need to read up to the split
point, and leave the after-split data sitting in the pipe/socket. Anyways...
that read could fail, meaning the split() would fail.

[ note: apr_brigade_partition() does not properly handle *failure* in a
  split operation; it assumes anything but ENOTIMPL is success ]

The end result is a partition() call that returns an error.

> > > If we do want to explicitly handle errors here, I think the way to do it
> > > is to move the calls to apr_brigade_partition() above the block that adds
> > > the range specifiers to the output stream and simply "continue;" if one
> > > of the calls fails for some wacky reason.
> >
> > Sure. We should also log an error because it means that something has broke
> > down during a bucket read. A failing CGI, for example.
> I'll buy this... if you can convince me that these calls to
> apr_brigade_partition() will ever have a reason to read from any bucket.

See above :-)

In fact, I am going to be changing mod_dav's interface to the back-end
repository to use bucket brigades(*). Let's say that a database backend
inserts a bucket that will read a BLOB out of a database. It knows the
length is N bytes, but it certainly doesn't want to read all that into
memory right away. So when it *does* read, it could fail (e.g. the database
connection timed out).

[ strictly speaking, you're right: partition() won't *read* in these
  scenarios, but the split() can easily fail. ]


(*) I already use this for DeltaV REPORT processing. I pass an ap_filter_t*
into the back-end, which then does some ap_fputstrs() to write into the
filter stack. I want to have repositories deliver their GET response into a
filter. Also, PROPFIND result generation can go into a filter rather than
accumulating in memory (ugh!) and then delivering it all. Should have a much
better working set this way, and reduce latency on the PROPFIND.

Greg Stein,

View raw message