couchdb-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Antony Blakey <antony.bla...@gmail.com>
Subject Re: Attachment Replication Problem - Bug Found
Date Sat, 16 May 2009 09:08:44 GMT

On 16/05/2009, at 12:59 PM, Antony Blakey wrote:

> and truncate the binary to the expected length. I'm not familiar  
> with ibrowse in terms of debugging this problem further.

The final mod I've ended up with is this, which deals with the ibrowse  
problem:

------------------------------------------------------------------------------

write_streamed_attachment(_Stream, _F, 0, SpAcc) ->
     {ok, SpAcc};
write_streamed_attachment(Stream, F, LenLeft, nil) ->
     Bin = F(),
     TruncatedBin = check_bin_length(LenLeft, Bin),
     {ok, StreamPointer} = couch_stream:write(Stream, TruncatedBin),
     write_streamed_attachment(Stream, F, LenLeft -  
size(TruncatedBin), StreamPointer);
write_streamed_attachment(Stream, F, LenLeft, SpAcc) ->
     Bin = F(),
     TruncatedBin = check_bin_length(LenLeft, Bin),
     {ok, _} = couch_stream:write(Stream, TruncatedBin),
     write_streamed_attachment(Stream, F, LenLeft -  
size(TruncatedBin), SpAcc).

check_bin_length(LenLeft, Bin) when size(Bin) > LenLeft ->
     <<ValidData:LenLeft/binary, Crap/binary>> = Bin,
     ?LOG_ERROR("write_streamed_attachment has written too much  
expected: ~p got: ~p tail: ~p", [LenLeft, size(Bin), Crap]),
     ValidData;
check_bin_length(_, Bin) -> Bin.

------------------------------------------------------------------------------

Interestingly, the problems occur at the exactly the same points  
during replication, and in each case the excess tail is <<"\r">>,  
which suggests to me a boundary condition processing a chunked  
response. It's probably not a problem creating the response because  
direct access using wget returns the right amount of data.

My replication still fails near the end, this time silently killing  
couchdb, but it's getting closer.

Antony Blakey
--------------------------
CTO, Linkuistics Pty Ltd
Ph: 0438 840 787

Always have a vision. Why spend your life making other people’s dreams?
  -- Orson Welles (1915-1985)


Mime
View raw message