couchdb-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Antony Blakey <antony.bla...@gmail.com>
Subject Re: svn commit: r775634 - /couchdb/branches/0.9.x/src/couchdb/couch_db.erl
Date Mon, 18 May 2009 00:53:00 GMT

On 18/05/2009, at 8:44 AM, Adam Kocoloski wrote:

> Sure, I agree. Consider this commit a stopgap until we're able to  
> pin down the source of the problem.  Antony, any chance you can make  
> that 100% reproducible case publicly available?

I'll ask. Apart from being 3.5G it also contains mainly non-public  
work-in-progress client data.

I may be able to make it available selectively with a commercial-in- 
confidence rider. Would that help? My replication is still unreliable  
so I need to track that down, and hence don't have time to fix the  
cause of this bug given that I have this workaround.

BTW: I've just moved to R13B, and I think the problem isn't occurring  
- I'm chasing it down. If that's the case then we can chalk it up to  
an erlang bug.

>
> Adam
>
> On May 17, 2009, at 5:10 PM, Damien Katz wrote:
>
>> +%% on rare occasions ibrowse seems to process a chunked response  
>> incorrectly
>> +%% and include an extra "\r" in the last chunk.  This code ensures  
>> that we
>> +%% truncate the downloaed attachment at the length specified in  
>> the metadata.
>>
>> That really sucks. We need to either fix ibrowse or drop it  
>> altogether.
>>
>> -Damien
>>
>> On May 17, 2009, at 8:29 AM, kocolosk@apache.org wrote:
>>
>>> Author: kocolosk
>>> Date: Sun May 17 12:29:22 2009
>>> New Revision: 775634
>>>
>>> URL: http://svn.apache.org/viewvc?rev=775634&view=rev
>>> Log:
>>> work around ibrowse giving response chunks too many bytes.  Thanks  
>>> Antony.
>>>
>>> Modified:
>>>  couchdb/branches/0.9.x/src/couchdb/couch_db.erl
>>>
>>> Modified: couchdb/branches/0.9.x/src/couchdb/couch_db.erl
>>> URL: http://svn.apache.org/viewvc/couchdb/branches/0.9.x/src/couchdb/couch_db.erl?rev=775634&r1=775633&r2=775634&view=diff
>>> = 
>>> = 
>>> = 
>>> = 
>>> = 
>>> = 
>>> = 
>>> = 
>>> = 
>>> = 
>>> ====================================================================
>>> --- couchdb/branches/0.9.x/src/couchdb/couch_db.erl (original)
>>> +++ couchdb/branches/0.9.x/src/couchdb/couch_db.erl Sun May 17  
>>> 12:29:22 2009
>>> @@ -649,12 +649,24 @@
>>>   {ok, SpAcc};
>>> write_streamed_attachment(Stream, F, LenLeft, nil) ->
>>>   Bin = F(),
>>> -    {ok, StreamPointer} = couch_stream:write(Stream, Bin),
>>> -    write_streamed_attachment(Stream, F, LenLeft - size(Bin),  
>>> StreamPointer);
>>> +    TruncatedBin = check_bin_length(LenLeft, Bin),
>>> +    {ok, SpAcc} = couch_stream:write(Stream, TruncatedBin),
>>> +    write_streamed_attachment(Stream, F, LenLeft -  
>>> size(TruncatedBin), SpAcc);
>>> write_streamed_attachment(Stream, F, LenLeft, SpAcc) ->
>>>   Bin = F(),
>>> -    {ok, _} = couch_stream:write(Stream, Bin),
>>> -    write_streamed_attachment(Stream, F, LenLeft - size(Bin),  
>>> SpAcc).
>>> +    TruncatedBin = check_bin_length(LenLeft, Bin),
>>> +    {ok, _} = couch_stream:write(Stream, TruncatedBin),
>>> +    write_streamed_attachment(Stream, F, LenLeft -  
>>> size(TruncatedBin), SpAcc).
>>> +
>>> +%% on rare occasions ibrowse seems to process a chunked response  
>>> incorrectly
>>> +%% and include an extra "\r" in the last chunk.  This code  
>>> ensures that we
>>> +%% truncate the downloaed attachment at the length specified in  
>>> the metadata.
>>> +check_bin_length(LenLeft, Bin) when size(Bin) > LenLeft ->
>>> +    <<ValidData:LenLeft/binary, Crap/binary>> = Bin,
>>> +    ?LOG_ERROR("write_streamed_attachment has written too much  
>>> expected: ~p" ++
>>> +        " got: ~p tail: ~p", [LenLeft, size(Bin), Crap]),
>>> +    ValidData;
>>> +check_bin_length(_, Bin) -> Bin.
>>>
>>> enum_docs_since_reduce_to_count(Reds) ->
>>>   couch_btree:final_reduce(
>>>
>>>
>>
>

Antony Blakey
--------------------------
CTO, Linkuistics Pty Ltd
Ph: 0438 840 787

The project was so plagued by politics and ego that when the engineers  
requested technical oversight, our manager hired a psychologist instead.
   -- Ron Avitzur


Mime
View raw message