Mailing-List: contact couchdb-user-help@incubator.apache.org; run by ezmlm
Precedence: bulk
Reply-To: couchdb-user@incubator.apache.org
Received-SPF: neutral (athena.apache.org: local policy)
Message-ID: <64a10fff0807181949t3fa2612fl6f61b372e5e119d8@mail.gmail.com>
Date: Fri, 18 Jul 2008 22:49:17 -0400
From: "Dean Landolt" <dean@deanlandolt.com>
To: couchdb-user@incubator.apache.org
Subject: Re: attachment file size limit?
In-Reply-To: <95D76033-7D1E-4232-BF5A-E2A2597ED418@apache.org>
MIME-Version: 1.0
Content-Type: multipart/alternative;
	boundary="----=_Part_44976_28423539.1216435757136"
References: <64a10fff0807181735m626c1fd9vc96c08622ee37b6@mail.gmail.com>
	 <95D76033-7D1E-4232-BF5A-E2A2597ED418@apache.org>

------=_Part_44976_28423539.1216435757136
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: 7bit
Content-Disposition: inline

On Fri, Jul 18, 2008 at 9:37 PM, Jan Lehnardt <jan@apache.org> wrote:

>
> On Jul 19, 2008, at 02:35, Dean Landolt wrote:
>
>  I seem to be running up an arbitrary file size limit in MochiWeb. For
>> smaller files I'm in good shape, but once I get upward of a few megs I get
>> this back from MochiWeb:
>>
>> {"error":"EXIT","reason":"{body_too_large,6099917}"}
>>
>> I poked around in the MochiWeb source and it looks like there's a setting
>> for this, but I haven't explored further yet.
>>
>> I tried to use chunked encoding but I seem to run into a socket error
>> issue
>> with httplib: 104, connection reset by peer. Seems like MochiWeb doesn't
>> like that very much, but that could just be httplib. Still trying to work
>> through that...
>>
>> In any event, has anyone pushed in large attachments with the new api? Is
>> there something I'm missing? Thanks...
>>
>
> The current limit for request sizes is 1MB (didn't we up that to 4GB?
> Christopher,
> maybe that got lost with recent MochiWeb updates?).
>
> You can set the size for yourself in src/mochiweb/mochiweb_request.erl
> -define(MAX_RECV_BODY, (1024*1024)). <- that is the line. Then
> recompile and you are good to go. I was able to push 2.5GB into an
> attachment.
>
> Please note that CouchDB holds the entire attachment in memory, even
> with a chunked request, before writing it to disk. This is a current design
> limitation and will be removed for 0.9. We are aware that this stinks, but
> so far the new API removed the need for base64 which was another
> performance killer. Streamed read and write operations will follow.
>
> I used curl to make successful chunked requests.
>
> Cheers
> Jan
> --
>

Done and done. I still can't get chunking to work w/ my httplib2
implementation but I don't care -- it's working beautifully otherwise. On
top of that, outbound streaming seems to be functioning nicely, at least for
the mp3s I dumped in to test (I'm running 0.9.0a676702 -- perhaps that
feature's already committed?). Thanks for your help again, Jan...

Dean

------=_Part_44976_28423539.1216435757136--