From dev-return-4232-apmail-couchdb-dev-archive=couchdb.apache.org@couchdb.apache.org Sat May 16 09:09:29 2009 Return-Path: Delivered-To: apmail-couchdb-dev-archive@www.apache.org Received: (qmail 1536 invoked from network); 16 May 2009 09:09:27 -0000 Received: from hermes.apache.org (HELO mail.apache.org) (140.211.11.3) by minotaur.apache.org with SMTP; 16 May 2009 09:09:27 -0000 Received: (qmail 65832 invoked by uid 500); 16 May 2009 09:09:25 -0000 Delivered-To: apmail-couchdb-dev-archive@couchdb.apache.org Received: (qmail 65746 invoked by uid 500); 16 May 2009 09:09:25 -0000 Mailing-List: contact dev-help@couchdb.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@couchdb.apache.org Delivered-To: mailing list dev@couchdb.apache.org Received: (qmail 65736 invoked by uid 99); 16 May 2009 09:09:25 -0000 Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230) by apache.org (qpsmtpd/0.29) with ESMTP; Sat, 16 May 2009 09:09:25 +0000 X-ASF-Spam-Status: No, hits=1.2 required=10.0 tests=FS_REPLICA,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (nike.apache.org: domain of antony.blakey@gmail.com designates 209.85.198.233 as permitted sender) Received: from [209.85.198.233] (HELO rv-out-0506.google.com) (209.85.198.233) by apache.org (qpsmtpd/0.29) with ESMTP; Sat, 16 May 2009 09:09:14 +0000 Received: by rv-out-0506.google.com with SMTP id g37so1088649rvb.35 for ; Sat, 16 May 2009 02:08:53 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:message-id:from:to :in-reply-to:content-type:content-transfer-encoding:mime-version :subject:date:references:x-mailer; bh=+uwE/bhUPncci63x68PwJMMn86YcwxMCRjNmIgXpKtE=; b=I4/cIMpEdCY2+3uByiaVIwtnFUy55NiUlkSqN5EdCgBFPH9XWnsvJhX5Pi3YCU6Ubb xRZtpmS/clKg6bzfna9AS8KJqHwzFZGcGvbRt72Y6abAPMspHpd5RHPW2jCqMXJcsn03 3SQrR8yl9njguaM7Nk+CjYUFjqIVR9OegOxT4= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=message-id:from:to:in-reply-to:content-type :content-transfer-encoding:mime-version:subject:date:references :x-mailer; b=mNh7Xp/G06vCLgXuh0Vn9Sc/qN1NC8EuMzbFaXS6PdGXGlBz3GniTfjj1QukzCshxV dEd6npPCsl6+5NQ8cga/VAm0am2Uy9t9W1yHiEtM4ZtVRxaFxyL9BNCBake+vrgsLBNa I7hSui8v5L514bZF6yYqvLCNRt5/68BR0nsxI= Received: by 10.140.142.11 with SMTP id p11mr1648749rvd.206.1242464932962; Sat, 16 May 2009 02:08:52 -0700 (PDT) Received: from ?192.168.0.11? ([202.137.112.11]) by mx.google.com with ESMTPS id b39sm582704rvf.9.2009.05.16.02.08.50 (version=TLSv1/SSLv3 cipher=RC4-MD5); Sat, 16 May 2009 02:08:52 -0700 (PDT) Message-Id: <9F12A09E-64D7-4703-A295-81B205655300@gmail.com> From: Antony Blakey To: dev@couchdb.apache.org In-Reply-To: Content-Type: text/plain; charset=WINDOWS-1252; format=flowed; delsp=yes Content-Transfer-Encoding: quoted-printable Mime-Version: 1.0 (Apple Message framework v930.4) Subject: Re: Attachment Replication Problem - Bug Found Date: Sat, 16 May 2009 17:08:44 +0800 References: <12669510-BD0F-45F3-8AC6-19872DF9071D@gmail.com> <25F55CAE-C44E-4416-AF39-C9037BA099FE@gmail.com> X-Mailer: Apple Mail (2.930.4) X-Virus-Checked: Checked by ClamAV on apache.org On 16/05/2009, at 12:59 PM, Antony Blakey wrote: > and truncate the binary to the expected length. I'm not familiar =20 > with ibrowse in terms of debugging this problem further. The final mod I've ended up with is this, which deals with the ibrowse =20= problem: = --------------------------------------------------------------------------= ---- write_streamed_attachment(_Stream, _F, 0, SpAcc) -> {ok, SpAcc}; write_streamed_attachment(Stream, F, LenLeft, nil) -> Bin =3D F(), TruncatedBin =3D check_bin_length(LenLeft, Bin), {ok, StreamPointer} =3D couch_stream:write(Stream, TruncatedBin), write_streamed_attachment(Stream, F, LenLeft - =20 size(TruncatedBin), StreamPointer); write_streamed_attachment(Stream, F, LenLeft, SpAcc) -> Bin =3D F(), TruncatedBin =3D check_bin_length(LenLeft, Bin), {ok, _} =3D couch_stream:write(Stream, TruncatedBin), write_streamed_attachment(Stream, F, LenLeft - =20 size(TruncatedBin), SpAcc). check_bin_length(LenLeft, Bin) when size(Bin) > LenLeft -> <> =3D Bin, ?LOG_ERROR("write_streamed_attachment has written too much =20 expected: ~p got: ~p tail: ~p", [LenLeft, size(Bin), Crap]), ValidData; check_bin_length(_, Bin) -> Bin. = --------------------------------------------------------------------------= ---- Interestingly, the problems occur at the exactly the same points =20 during replication, and in each case the excess tail is <<"\r">>, =20 which suggests to me a boundary condition processing a chunked =20 response. It's probably not a problem creating the response because =20 direct access using wget returns the right amount of data. My replication still fails near the end, this time silently killing =20 couchdb, but it's getting closer. Antony Blakey -------------------------- CTO, Linkuistics Pty Ltd Ph: 0438 840 787 Always have a vision. Why spend your life making other people=92s = dreams? -- Orson Welles (1915-1985)