From user-return-19723-apmail-couchdb-user-archive=couchdb.apache.org@couchdb.apache.org Wed Feb 1 16:55:38 2012 Return-Path: X-Original-To: apmail-couchdb-user-archive@www.apache.org Delivered-To: apmail-couchdb-user-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id D0E0F965E for ; Wed, 1 Feb 2012 16:55:38 +0000 (UTC) Received: (qmail 70234 invoked by uid 500); 1 Feb 2012 16:55:37 -0000 Delivered-To: apmail-couchdb-user-archive@couchdb.apache.org Received: (qmail 70193 invoked by uid 500); 1 Feb 2012 16:55:36 -0000 Mailing-List: contact user-help@couchdb.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@couchdb.apache.org Delivered-To: mailing list user@couchdb.apache.org Received: (qmail 70185 invoked by uid 99); 1 Feb 2012 16:55:35 -0000 Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 01 Feb 2012 16:55:35 +0000 X-ASF-Spam-Status: No, hits=2.1 required=5.0 tests=DATE_IN_PAST_96_XX,SPF_HELO_PASS,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (nike.apache.org: domain of gcdcu-couchdb-user@m.gmane.org designates 80.91.229.3 as permitted sender) Received: from [80.91.229.3] (HELO plane.gmane.org) (80.91.229.3) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 01 Feb 2012 16:55:26 +0000 Received: from list by plane.gmane.org with local (Exim 4.69) (envelope-from ) id 1RsdSZ-0003c7-Mt for user@couchdb.apache.org; Wed, 01 Feb 2012 17:55:04 +0100 Received: from attensity.com ([4.59.218.31]) by main.gmane.org with esmtp (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Wed, 01 Feb 2012 17:55:03 +0100 Received: from schidester by attensity.com with local (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Wed, 01 Feb 2012 17:55:03 +0100 X-Injected-Via-Gmane: http://gmane.org/ To: user@couchdb.apache.org From: Shawn Chidester Subject: CouchDB =?utf-8?b?X2J1bGtfZG9jcw==?= allocating far more memory than needed? Date: Fri, 27 Jan 2012 21:05:32 +0000 (UTC) Lines: 33 Message-ID: Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-Complaints-To: usenet@dough.gmane.org X-Gmane-NNTP-Posting-Host: sea.gmane.org User-Agent: Loom/3.14 (http://gmane.org/) X-Loom-IP: 69.27.13.199 (Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/535.7 (KHTML, like Gecko) Chrome/16.0.912.77 Safari/535.7) X-Virus-Checked: Checked by ClamAV on apache.org I'm attempting to post a single 150 MB document to my couchdb server. The application that does the post usually has more than one document to post so it always uses _bulk_docs, even for one document. According to the crash message the couchdb server (or erlang) is tring to allocate 4 GB or memory for this 150 MB post. Here is the console output when I run the couchdb server in the foreground (with log level set to debug so we can see the post header): [debug] [<0.783.0>] 'POST' /dbname/_bulk_docs {1,1} from "--manually_omitted--" Headers: [{'Accept-Encoding',"gzip"}, {'Accept-Language',"en-US,*"}, {'Authorization',"Basic --manually_omitted--"}, {'Connection',"Keep-Alive"}, {'Content-Length',"153786246"}, {'Content-Type',"application/json;charset=UTF-8"}, {'Host',"--manually_omitted--"}, {'User-Agent',"--manually_omitted--"}] [debug] [<0.783.0>] OAuth Params: [] Crash dump was written to: erl_crash.dump eheap_alloc: Cannot allocate 4454408120 bytes of memory (of type "heap"). Aborted Can anyone confirm and/or explain this behavior? Thanks, Shawn