couchdb-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Adam Kocoloski <kocol...@apache.org>
Subject Re: svn commit: r775634 - /couchdb/branches/0.9.x/src/couchdb/couch_db.erl
Date Mon, 18 May 2009 00:58:35 GMT
Commercial-in-confidence works for me.  Thanks for the tip about the  
OTP release version, too.  Best,

Adam

On May 17, 2009, at 8:53 PM, Antony Blakey wrote:

>
> On 18/05/2009, at 8:44 AM, Adam Kocoloski wrote:
>
>> Sure, I agree. Consider this commit a stopgap until we're able to  
>> pin down the source of the problem.  Antony, any chance you can  
>> make that 100% reproducible case publicly available?
>
> I'll ask. Apart from being 3.5G it also contains mainly non-public  
> work-in-progress client data.
>
> I may be able to make it available selectively with a commercial-in- 
> confidence rider. Would that help? My replication is still  
> unreliable so I need to track that down, and hence don't have time  
> to fix the cause of this bug given that I have this workaround.
>
> BTW: I've just moved to R13B, and I think the problem isn't  
> occurring - I'm chasing it down. If that's the case then we can  
> chalk it up to an erlang bug.
>
>>
>> Adam
>>
>> On May 17, 2009, at 5:10 PM, Damien Katz wrote:
>>
>>> +%% on rare occasions ibrowse seems to process a chunked response  
>>> incorrectly
>>> +%% and include an extra "\r" in the last chunk.  This code  
>>> ensures that we
>>> +%% truncate the downloaed attachment at the length specified in  
>>> the metadata.
>>>
>>> That really sucks. We need to either fix ibrowse or drop it  
>>> altogether.
>>>
>>> -Damien
>>>
>>> On May 17, 2009, at 8:29 AM, kocolosk@apache.org wrote:
>>>
>>>> Author: kocolosk
>>>> Date: Sun May 17 12:29:22 2009
>>>> New Revision: 775634
>>>>
>>>> URL: http://svn.apache.org/viewvc?rev=775634&view=rev
>>>> Log:
>>>> work around ibrowse giving response chunks too many bytes.   
>>>> Thanks Antony.
>>>>
>>>> Modified:
>>>> couchdb/branches/0.9.x/src/couchdb/couch_db.erl
>>>>
>>>> Modified: couchdb/branches/0.9.x/src/couchdb/couch_db.erl
>>>> URL: http://svn.apache.org/viewvc/couchdb/branches/0.9.x/src/couchdb/couch_db.erl?rev=775634&r1=775633&r2=775634&view=diff
>>>> = 
>>>> = 
>>>> = 
>>>> = 
>>>> = 
>>>> = 
>>>> = 
>>>> = 
>>>> = 
>>>> = 
>>>> = 
>>>> ===================================================================
>>>> --- couchdb/branches/0.9.x/src/couchdb/couch_db.erl (original)
>>>> +++ couchdb/branches/0.9.x/src/couchdb/couch_db.erl Sun May 17  
>>>> 12:29:22 2009
>>>> @@ -649,12 +649,24 @@
>>>>  {ok, SpAcc};
>>>> write_streamed_attachment(Stream, F, LenLeft, nil) ->
>>>>  Bin = F(),
>>>> -    {ok, StreamPointer} = couch_stream:write(Stream, Bin),
>>>> -    write_streamed_attachment(Stream, F, LenLeft - size(Bin),  
>>>> StreamPointer);
>>>> +    TruncatedBin = check_bin_length(LenLeft, Bin),
>>>> +    {ok, SpAcc} = couch_stream:write(Stream, TruncatedBin),
>>>> +    write_streamed_attachment(Stream, F, LenLeft -  
>>>> size(TruncatedBin), SpAcc);
>>>> write_streamed_attachment(Stream, F, LenLeft, SpAcc) ->
>>>>  Bin = F(),
>>>> -    {ok, _} = couch_stream:write(Stream, Bin),
>>>> -    write_streamed_attachment(Stream, F, LenLeft - size(Bin),  
>>>> SpAcc).
>>>> +    TruncatedBin = check_bin_length(LenLeft, Bin),
>>>> +    {ok, _} = couch_stream:write(Stream, TruncatedBin),
>>>> +    write_streamed_attachment(Stream, F, LenLeft -  
>>>> size(TruncatedBin), SpAcc).
>>>> +
>>>> +%% on rare occasions ibrowse seems to process a chunked response  
>>>> incorrectly
>>>> +%% and include an extra "\r" in the last chunk.  This code  
>>>> ensures that we
>>>> +%% truncate the downloaed attachment at the length specified in  
>>>> the metadata.
>>>> +check_bin_length(LenLeft, Bin) when size(Bin) > LenLeft ->
>>>> +    <<ValidData:LenLeft/binary, Crap/binary>> = Bin,
>>>> +    ?LOG_ERROR("write_streamed_attachment has written too much  
>>>> expected: ~p" ++
>>>> +        " got: ~p tail: ~p", [LenLeft, size(Bin), Crap]),
>>>> +    ValidData;
>>>> +check_bin_length(_, Bin) -> Bin.
>>>>
>>>> enum_docs_since_reduce_to_count(Reds) ->
>>>>  couch_btree:final_reduce(
>>>>
>>>>
>>>
>>
>
> Antony Blakey
> --------------------------
> CTO, Linkuistics Pty Ltd
> Ph: 0438 840 787
>
> The project was so plagued by politics and ego that when the  
> engineers requested technical oversight, our manager hired a  
> psychologist instead.
>  -- Ron Avitzur
>


Mime
View raw message