trafficserver-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From James Peach <jamespe...@me.com>
Subject Re: inconsistent read IOBuffer results
Date Tue, 03 Jan 2012 02:47:50 GMT
On 02/01/2012, at 6:13 PM, Brian Geffon wrote:

> My last email was incorrect, it will correctly return the pointer to the
> memory of the block, but avail will not be touched and you will have no
> idea how much data could have been read. You actually don't even need to
> call TSIOBufferBlockReadStart(), TSIOBufferReadAvail() should just be
> equal to the sum of all of the calls to the TSIOBufferBlockReadAvail().

Yes, that's exactly the invariant that is being violated. I can sometimes end up with TSIOBufferReadAvail()
claiming there are more bytes than are actually available from the buffer.

I'm inside the TSVConnRead() continuation, responding to TS_EVENT_VCONN_READ_READY, so I don't
*think* that I need to do any additional locking. It might be an internal race, but I have
slightly hacked up my ATS tree, so it's also possible that I caused a stale value to be reported
by TSIOBufferReadAvail().

> The following is untested but it appears that it should give what you want:
> 
> block = TSIOBufferStart(buffer);
>   while (block) {
>  count += TSIOBufferBlockReadAvail(block,reader);
>  block = TSIOBufferBlockNext(block);
>   }

That's a more succinct version of the code I posted, right?

> 
> Brian
> 
> 
> 
> On 1/2/12 6:04 PM, "Brian Geffon" <briangeffon@gmail.com> wrote:
> 
>> I dont think this is a bug, your usage of TSIOBufferBlockReadStart()
>> is incorrect. The third parameter which you call nbytes is set to the
>> number of bytes remaining in the buffer block after the read. Since
>> you're setting it to zero before the call to
>> TSIOBufferBlockReadStart() it's actually not going to read anything...
>> 
>> if (avail)
>>   *avail = blk->read_avail();
>> 
>> So since it's not getting the correct size of the block, and thus will
>> not reading anything anyway because before it tries to read the block
>> it does:
>> 
>> if (avail) {
>>     *avail -= reader->start_offset;
>>     if (*avail < 0) {
>>       *avail = 0;
>>     }
>>   }
>> 
>> So it's never going to touch avail. So that means nbytes be set to
>> TSIOBufferBlockReadAvail() before the call
>> 
>> int64_t nbytes = TSIOBufferBlockReadAvail( ... )
>> int64_t nbytes_remaining = nbytes;
>> 
>> and then, nbytes_remaining will be the number of bytes remaining to be
>> read, which should actually be 0 after the call to
>> TSIOBufferBlockReadStart(), So the total bytes contained in the block
>> was :
>> 
>> ptr = TSIOBufferBlockReadStart(block, reader, &nbytes_remaining);
>> if(ptr) {
>> int64_t bytes_in_block = nbytes - nbytes_remaining; // that's how
>> much we actually were able to read
>> count += bytes_in_block;
>> }
>> 
>> Does that help?
>> 
>> On Mon, Jan 2, 2012 at 5:45 PM, James Peach <jamespeach@me.com> wrote:
>>> On 02/01/2012, at 1:18 PM, James Peach wrote:
>>> 
>>>> On 02/01/2012, at 11:30 AM, Brian Geffon wrote:
>>>> 
>>>>> I think you might want TSIOBufferBlockReadAvail and not
>>>>> TSIOBufferReaderAvail.
>>>> 
>>>> Hmm, so my code is assuming that all the data is in the first buffer
>>>> block. It sounds like that it not guaranteed and that I ought to be
>>>> calling TSIOBufferBlockNext() if the first buffer block doesn't have
>>>> all the data.
>>> 
>>> After some more testing, I think that this is a bug.
>>> TSIOBufferReaderAvail() returns 43, but there are subsequently 0 bytes
>>> available. Iterating the buffer blocks as below confirms this. My
>>> understanding of TSIOBufferReaderAvail() and the assumption of other
>>> usages of it that I have seen is that the data in teh buffer blocks
>>> should all add up to the available count from the buffer reader. I
>>> haven't seen any indication that TSIOBufferReaderAvail() is advisory.
>>> 
>>> static size_t
>>> count_bytes_available(
>>>       TSIOBuffer          buffer,
>>>       TSIOBufferReader    reader)
>>> {
>>>   TSIOBufferBlock block;
>>>   size_t count = 0;
>>> 
>>>   block = TSIOBufferStart(buffer);
>>>   while (block) {
>>>       const char * ptr;
>>>       int64_t nbytes = 0;
>>> 
>>>       ptr = TSIOBufferBlockReadStart(block, reader, &nbytes);
>>>       if (ptr && nbytes) {
>>>           count += nbytes;
>>>       }
>>> 
>>>       block = TSIOBufferBlockNext(block);
>>>   }
>>> 
>>>   return count;
>>> }
>>> 
>>>> 
>>>> thanks,
>>>> James
>>>> 
>>>>> 
>>>>> Brian
>>>>> ________________________________________
>>>>> From: James Peach [jamespeach@me.com]
>>>>> Sent: Saturday, December 31, 2011 10:07 PM
>>>>> To: dev@trafficserver.apache.org
>>>>> Subject: inconsistent read IOBuffer results
>>>>> 
>>>>> Hi all,
>>>>> 
>>>>> In my proxy code, I have something that looks roughly like this:
>>>>> 
>>>>>      if (TSIOBufferReaderAvail(reader) >= 10) {
>>>>>              blk = TSIOBufferStart(buffer);
>>>>>              ptr = (const uint8_t *)TSIOBufferBlockReadStart(blk,
>>>>> reader, &nbytes);
>>>>> 
>>>>>              TSReleaseAssert(nbytes >= 10);
>>>>>      }
>>>>> 
>>>>> Occasionally, the assertion will trigger; is that something that I
>>>>> should expect and handle?
>>>>> 
>>>>> cheers,
>>>>> James
>>>> 
>>> 
> 


Mime
View raw message