Mailing-List: contact dev-help@trafficserver.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@trafficserver.apache.org
Received-SPF: pass (athena.apache.org: domain of bgeffon@linkedin.com
 designates 69.28.149.81 as permitted sender)
DomainKey-Signature: s=prod; d=linkedin.com; c=nofws; q=dns;
  h=X-IronPort-AV:Received:From:To:Subject:Thread-Topic:
   Thread-Index:Date:Message-ID:In-Reply-To:Accept-Language:
   Content-Language:X-MS-Has-Attach:X-MS-TNEF-Correlator:
   x-originating-ip:Content-Type:Content-ID:
   Content-Transfer-Encoding:MIME-Version;
  b=JHP8XVuzlY/qhV9B5VOhIw7HJ4gIEP/q5NNsvMBhFNHIwEyVv+dxjRpX
   VfDLUxwLs+nPX1T0Ou9E2jFoU5WmDY4VSE8LqgLtSROumiV/TfY8xjW+N
   E/sHyvaDLM2QRgn;
From: Brian Geffon <bgeffon@linkedin.com>
To: "dev@trafficserver.apache.org" <dev@trafficserver.apache.org>
Subject: Re: inconsistent read IOBuffer results
Thread-Topic: inconsistent read IOBuffer results
Thread-Index: AQHMyEvDJh2XY88gwUC6QP7lJlsLV5X5dfR0gACnlwCAAEqugIAABWEA//98VAA=
Date: Tue, 3 Jan 2012 02:13:13 +0000
Message-ID: <CB27A32F.3FAC%bgeffon@linkedin.com>
In-Reply-To: 
 <CAFKFyq61Wd7LNnnMFAE61nL=meDUi++wVPbTQH3WuR9WNYyooA@mail.gmail.com>
Accept-Language: en-US
Content-Language: en-US
Content-Type: text/plain; charset="us-ascii"
Content-ID: <D2F682A0F96E484989236B9E3173B16C@linkedin.com>
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0

My last email was incorrect, it will correctly return the pointer to the
memory of the block, but avail will not be touched and you will have no
idea how much data could have been read. You actually don't even need to
call TSIOBufferBlockReadStart(), TSIOBufferReadAvail() should just be
equal to the sum of all of the calls to the TSIOBufferBlockReadAvail().
The following is untested but it appears that it should give what you want:

block =3D TSIOBufferStart(buffer);
   while (block) {
  count +=3D TSIOBufferBlockReadAvail(block,reader);
  block =3D TSIOBufferBlockNext(block);
   }

Brian


On 1/2/12 6:04 PM, "Brian Geffon" <briangeffon@gmail.com> wrote:

>I dont think this is a bug, your usage of TSIOBufferBlockReadStart()
>is incorrect. The third parameter which you call nbytes is set to the
>number of bytes remaining in the buffer block after the read. Since
>you're setting it to zero before the call to
>TSIOBufferBlockReadStart() it's actually not going to read anything...
>
>  if (avail)
>    *avail =3D blk->read_avail();
>
>So since it's not getting the correct size of the block, and thus will
>not reading anything anyway because before it tries to read the block
>it does:
>
>  if (avail) {
>      *avail -=3D reader->start_offset;
>      if (*avail < 0) {
>        *avail =3D 0;
>      }
>    }
>
>So it's never going to touch avail. So that means nbytes be set to
>TSIOBufferBlockReadAvail() before the call
>
>int64_t nbytes =3D TSIOBufferBlockReadAvail( ... )
>int64_t nbytes_remaining =3D nbytes;
>
>and then, nbytes_remaining will be the number of bytes remaining to be
>read, which should actually be 0 after the call to
>TSIOBufferBlockReadStart(), So the total bytes contained in the block
>was :
>
> ptr =3D TSIOBufferBlockReadStart(block, reader, &nbytes_remaining);
> if(ptr) {
>  int64_t bytes_in_block =3D nbytes - nbytes_remaining; // that's how
>much we actually were able to read
>  count +=3D bytes_in_block;
> }
>
>Does that help?
>
>On Mon, Jan 2, 2012 at 5:45 PM, James Peach <jamespeach@me.com> wrote:
>> On 02/01/2012, at 1:18 PM, James Peach wrote:
>>
>>> On 02/01/2012, at 11:30 AM, Brian Geffon wrote:
>>>
>>>> I think you might want TSIOBufferBlockReadAvail and not
>>>>TSIOBufferReaderAvail.
>>>
>>> Hmm, so my code is assuming that all the data is in the first buffer
>>>block. It sounds like that it not guaranteed and that I ought to be
>>>calling TSIOBufferBlockNext() if the first buffer block doesn't have
>>>all the data.
>>
>> After some more testing, I think that this is a bug.
>>TSIOBufferReaderAvail() returns 43, but there are subsequently 0 bytes
>>available. Iterating the buffer blocks as below confirms this. My
>>understanding of TSIOBufferReaderAvail() and the assumption of other
>>usages of it that I have seen is that the data in teh buffer blocks
>>should all add up to the available count from the buffer reader. I
>>haven't seen any indication that TSIOBufferReaderAvail() is advisory.
>>
>> static size_t
>> count_bytes_available(
>>        TSIOBuffer          buffer,
>>        TSIOBufferReader    reader)
>> {
>>    TSIOBufferBlock block;
>>    size_t count =3D 0;
>>
>>    block =3D TSIOBufferStart(buffer);
>>    while (block) {
>>        const char * ptr;
>>        int64_t nbytes =3D 0;
>>
>>        ptr =3D TSIOBufferBlockReadStart(block, reader, &nbytes);
>>        if (ptr && nbytes) {
>>            count +=3D nbytes;
>>        }
>>
>>        block =3D TSIOBufferBlockNext(block);
>>    }
>>
>>    return count;
>> }
>>
>>>
>>> thanks,
>>> James
>>>
>>>>
>>>> Brian
>>>> ________________________________________
>>>> From: James Peach [jamespeach@me.com]
>>>> Sent: Saturday, December 31, 2011 10:07 PM
>>>> To: dev@trafficserver.apache.org
>>>> Subject: inconsistent read IOBuffer results
>>>>
>>>> Hi all,
>>>>
>>>> In my proxy code, I have something that looks roughly like this:
>>>>
>>>>       if (TSIOBufferReaderAvail(reader) >=3D 10) {
>>>>               blk =3D TSIOBufferStart(buffer);
>>>>               ptr =3D (const uint8_t *)TSIOBufferBlockReadStart(blk,
>>>>reader, &nbytes);
>>>>
>>>>               TSReleaseAssert(nbytes >=3D 10);
>>>>       }
>>>>
>>>> Occasionally, the assertion will trigger; is that something that I
>>>>should expect and handle?
>>>>
>>>> cheers,
>>>> James
>>>
>>