Return-Path: Delivered-To: apmail-couchdb-dev-archive@www.apache.org Received: (qmail 6684 invoked from network); 16 Jan 2009 10:16:10 -0000 Received: from hermes.apache.org (HELO mail.apache.org) (140.211.11.2) by minotaur.apache.org with SMTP; 16 Jan 2009 10:16:10 -0000 Received: (qmail 67302 invoked by uid 500); 16 Jan 2009 10:16:09 -0000 Delivered-To: apmail-couchdb-dev-archive@couchdb.apache.org Received: (qmail 67259 invoked by uid 500); 16 Jan 2009 10:16:08 -0000 Mailing-List: contact dev-help@couchdb.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@couchdb.apache.org Delivered-To: mailing list dev@couchdb.apache.org Received: (qmail 67248 invoked by uid 99); 16 Jan 2009 10:16:08 -0000 Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 16 Jan 2009 02:16:08 -0800 X-ASF-Spam-Status: No, hits=2.4 required=10.0 tests=NORMAL_HTTP_TO_IP,SPF_PASS,URIBL_RHS_DOB,WEIRD_PORT X-Spam-Check-By: apache.org Received-SPF: pass (nike.apache.org: domain of bchesneau@gmail.com designates 72.14.220.154 as permitted sender) Received: from [72.14.220.154] (HELO fg-out-1718.google.com) (72.14.220.154) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 16 Jan 2009 10:16:00 +0000 Received: by fg-out-1718.google.com with SMTP id 13so779203fge.3 for ; Fri, 16 Jan 2009 02:15:40 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:in-reply-to:references :date:message-id:subject:from:to:content-type; bh=uYveF2LbGljTAA5g4hRg2qp7OiRCnQgBaiaLrRHHXEA=; b=RUExWh5v2SwEUZkXtPApQV4bHTVFOVqroWRCiUJZvDPGkP+eSpNPSYPO5+2seJXv5o pfvG11I+BTsc/E0IkcvmaterR9Mulko8xon3hVIpO8Wu+1UsodSFcHszY8fU9guqKQCQ 729OezBYMNS0xgZpssDyc00gSUls1fFrBAPXg= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type; b=Xj2YLaOZCk9L1Y5ppiqStEDBPZXo6239NPFKU+WeDUsD7SYYel0YmMExLVtZq122HL a5LbxagvrOY2scO7+NI8tN0fmTOQUlgd1IbkAhbNMHPtAcLDQMm4b4QtTusimECpEil3 nnrxssVH5Ks7XbUhn0t+L/na6bGCKEfEc5GF8= MIME-Version: 1.0 Received: by 10.86.93.17 with SMTP id q17mr1935689fgb.8.1232100940343; Fri, 16 Jan 2009 02:15:40 -0800 (PST) In-Reply-To: <505B978F-8CB9-43FE-A593-679C94BC7DDD@apache.org> References: <505B978F-8CB9-43FE-A593-679C94BC7DDD@apache.org> Date: Fri, 16 Jan 2009 11:15:40 +0100 Message-ID: Subject: Re: streaming attachments writes From: Benoit Chesneau To: dev@couchdb.apache.org Content-Type: multipart/mixed; boundary=000e0cd25ceae06bd9046096da2e X-Virus-Checked: Checked by ClamAV on apache.org --000e0cd25ceae06bd9046096da2e Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit On Fri, Jan 16, 2009 at 1:07 AM, Damien Katz wrote: > I checked in streaming attachment writes for attachment uploads (ie PUT > /db/docid/attachment.txt ...). This allows that huge files can be uploaded > without CouchDB buffering it to memory, making is possible to upload huge > attachments. > > Unfortunately, we don't yet stream the attachments for replication w and you > can't update an attachment and the document json in single request yet, so > this is of limited use for now. That will require sending documents in http > multi-part, and support for that is on the to-do list. > > -Damien > Thanks for this update :) I've just tested this morning, here on a 2GB of ram machine, it didn''t work with curl in chunked transfert encoding. It doesn't work either in chunked mode with one script I have. However it works with this script based on py-restclient (attached) in normal mode. Thanks for this progress :) - benoit --000e0cd25ceae06bd9046096da2e Content-Type: text/x-python; charset=US-ASCII; name="test.py" Content-Disposition: attachment; filename="test.py" Content-Transfer-Encoding: base64 X-Attachment-Id: f_fq0p2l5e0 CmltcG9ydCBvcwpmcm9tIHJlc3RjbGllbnQgaW1wb3J0IFJlc291cmNlCgpmID0gb3BlbigndGVz dDY0MCcpCgpyZXMgPSBSZXNvdXJjZSgnaHR0cDovLzEyNy4wLjAuMTo1OTg0L3Rlc3QnKQoKY29u dGVudCA9IHJlcy5wdXQoJy9ibGFoL3Rlc3Q2NDAnLCBwYXlsb2FkPWYsIGhlYWRlcnM9ewogICAg J0FjY2VwdCc6ICdhcHBsaWNhdGlvbi9qc29uJywKICAgICdDb25uZWN0aW9uJzogJ2tlZXAtYWxp dmUnLAogICAgJ0NvbnRlbnQtVHlwZSc6ICdhcHBsaWNhdGlvbi9vY3RldC1zdHJlYW0nLAogICAg J0tlZXAtQWxpdmUnOiAnMzAwJywKICAgICdDb250ZW50LUxlbmd0aCc6IG9zLnBhdGguZ2V0c2l6 ZSgndGVzdC5tcDMnKX0sIHJldj0iMzMzMzY0NTYwNSIpCgpwcmludCBjb250ZW50Cg== --000e0cd25ceae06bd9046096da2e--