From user-return-21724-apmail-couchdb-user-archive=couchdb.apache.org@couchdb.apache.org Wed Aug 8 17:07:48 2012 Return-Path: X-Original-To: apmail-couchdb-user-archive@www.apache.org Delivered-To: apmail-couchdb-user-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 259C4DBAC for ; Wed, 8 Aug 2012 17:07:48 +0000 (UTC) Received: (qmail 95985 invoked by uid 500); 8 Aug 2012 17:07:46 -0000 Delivered-To: apmail-couchdb-user-archive@couchdb.apache.org Received: (qmail 95906 invoked by uid 500); 8 Aug 2012 17:07:46 -0000 Mailing-List: contact user-help@couchdb.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@couchdb.apache.org Delivered-To: mailing list user@couchdb.apache.org Received: (qmail 95895 invoked by uid 99); 8 Aug 2012 17:07:46 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 08 Aug 2012 17:07:46 +0000 X-ASF-Spam-Status: No, hits=-0.0 required=5.0 tests=RCVD_IN_DNSWL_NONE,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (athena.apache.org: domain of jens@couchbase.com designates 206.225.164.28 as permitted sender) Received: from [206.225.164.28] (HELO EXHUB020-1.exch020.serverdata.net) (206.225.164.28) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 08 Aug 2012 17:07:40 +0000 Received: from EXVMBX020-1.exch020.serverdata.net ([169.254.4.56]) by EXHUB020-1.exch020.serverdata.net ([206.225.164.28]) with mapi; Wed, 8 Aug 2012 10:07:18 -0700 From: Jens Alfke To: "user@couchdb.apache.org" CC: Robert Newson , Pieter van der Eems Date: Wed, 8 Aug 2012 10:07:18 -0700 Subject: Re: function_clause error in HTTP request Thread-Topic: function_clause error in HTTP request Thread-Index: Ac11iEN5HcFU5xr/RxqdxSCt6isRmQ== Message-ID: <85C1E8B6-781E-4880-B919-72BD155F96A2@couchbase.com> References: <12119867-1D77-4943-B87C-B8369C7005C8@couchbase.com> <80877B30-A67E-4FA6-B19A-93087D80BF55@couchbase.com> In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: acceptlanguage: en-US Content-Type: text/plain; charset="Windows-1252" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-Virus-Checked: Checked by ClamAV on apache.org I've figured this out, thanks to Robert Newson looking at a TCP dump Pieter= van der Eems sent him. It turns out to be an issue with CouchDB that I alr= eady knew about but had forgotten would bite in this particular circumstanc= e. Specifically, CouchDB isn't associating the MIME bodies with the attachm= ents correctly; it gets them mixed up. As a result it gets confused about t= he lengths and blows up. The issue is with CouchDB's multipart support, specifically the way in whic= h it matches MIME bodies to attachment names. The IMHO correct way to do th= is would be to look at the filename in the Content-Disposition header, and = this is in fact what TouchDB generates: Content-Disposition: attachment; filename=3D"20120808-092628.png" But CouchDB ignores this header. Instead it assumes that the order in which= the MIME bodies appear matches the order in which the attachment objects a= ppear in the _attachments object. The problem with this is that JSON objects (dictionaries) are _not_ ordered= collections. I know that Erlang's implementation of them (as linked lists = of key/value pairs) happens to be ordered, and I think some JavaScript impl= ementations have the side effect of preserving order; but in many languages= these are implemented as hash tables and genuinely unordered. So when TouchDB serializes the NSDictionary object representing the attachm= ents, it has _no idea_ in what order the JSON encoder will write the keys. = This means it can't comply with CouchDB's ordering requirement because it d= oesn't know what order in which to write out the attachments. I believe I a= m going to have to work around this by using a custom JSON encoder that I c= an make write out dictionary entries in a known (sorted?) order. I've filed this as COUCHDB-1521. As I said, I can work around it, but I rea= lly think this should be fixed as it's a hurdle for interoperability. (Ironically I ran into the flip side of this issue last year and filed a bu= g on it (COUCHDB-1368): when _receiving_ a multipart body from CouchDB, it'= s difficult to match attachments with their MIME bodies because CouchDB doe= sn't put any headers into the MIME bodies to indicate filenames; the only c= lue is the ordering of the entries in the _attachments dictionary, and that= ordering is lost when Cocoa's JSON parser converts it into an NSDictionary= object.) =97Jens=