From user-return-7110-apmail-couchdb-user-archive=couchdb.apache.org@couchdb.apache.org Mon Oct 26 20:27:14 2009 Return-Path: Delivered-To: apmail-couchdb-user-archive@www.apache.org Received: (qmail 7340 invoked from network); 26 Oct 2009 20:27:14 -0000 Received: from hermes.apache.org (HELO mail.apache.org) (140.211.11.3) by minotaur.apache.org with SMTP; 26 Oct 2009 20:27:14 -0000 Received: (qmail 2060 invoked by uid 500); 26 Oct 2009 20:27:13 -0000 Delivered-To: apmail-couchdb-user-archive@couchdb.apache.org Received: (qmail 1987 invoked by uid 500); 26 Oct 2009 20:27:13 -0000 Mailing-List: contact user-help@couchdb.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@couchdb.apache.org Delivered-To: mailing list user@couchdb.apache.org Received: (qmail 1977 invoked by uid 99); 26 Oct 2009 20:27:13 -0000 Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 26 Oct 2009 20:27:13 +0000 X-ASF-Spam-Status: No, hits=2.7 required=10.0 tests=HTML_MESSAGE,RCVD_IN_DNSWL_LOW,SPF_PASS,WEIRD_PORT X-Spam-Check-By: apache.org Received-SPF: pass (nike.apache.org: domain of richinsr@mac.com designates 17.148.16.92 as permitted sender) Received: from [17.148.16.92] (HELO asmtpout017.mac.com) (17.148.16.92) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 26 Oct 2009 20:27:03 +0000 MIME-version: 1.0 Content-type: multipart/alternative; boundary="Boundary_(ID_dYU4SjAquQEUmNC0lR90Vw)" Received: from [192.168.67.229] ([69.241.19.12]) by asmtp017.mac.com (Sun Java(tm) System Messaging Server 6.3-8.01 (built Dec 16 2008; 32bit)) with ESMTPSA id <0KS5008E923Y7920@asmtp017.mac.com> for user@couchdb.apache.org; Mon, 26 Oct 2009 13:26:26 -0700 (PDT) Message-id: From: Ryan Richins To: user@couchdb.apache.org Subject: Size of view file Date: Mon, 26 Oct 2009 16:24:39 -0400 X-Mailer: Apple Mail (2.936) X-Virus-Checked: Checked by ClamAV on apache.org --Boundary_(ID_dYU4SjAquQEUmNC0lR90Vw) Content-type: text/plain; charset=US-ASCII; format=flowed; delsp=yes Content-transfer-encoding: 7BIT I am working on a project where i have 12k documents and the size of the db is 11MB but the view file is is over 4GB. Obviously I am doing something wrong with my views to make the file so large. I was hoping to get some input as to where my problem might be. Running couchdb 0.90 Each document has 3 attributes one of which is 'User Agent'. For each attribute I have the following views defined "by__total_date" and "by_ _created_at". Below is the code for the 2 views that deal with User Agent. The same code is used to define the views for the other 2 attributes except doc.user_agent is replace by doc. My guess is the problem lies somewhere in the "by__total_date" since every other view I have returns NULL for the value. #1 by_ua_total_date ----------------- MAP: function(doc) { var val = {}; datetime = doc.created_at; year = parseInt(datetime.substr(0, 4)); month = parseInt(datetime.substr(5, 2), 10); day = parseInt(datetime.substr(8, 2), 10); val[doc.user_agent] = 1; emit([year, month, day], val ); } REDUCE: function (keys, values, rereduce) { var rv = {}; for (i in values) { var value = values[i]; for (k in value) { rv[k] = (rv[k] || 0) + value[k]; } } return rv; } EXAMPLE OUTPUT (Key, Value) [2009, 9, 6], {Mozilla/5.0 (iPod; U; CPU iPhone OS 2_2_1 like Mac OS X; en-us) AppleWebKit/525.18.1 (KHTML, like Gecko) Version/3.1.1 Mobile/5H11a Safari/525.20: 5, Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1; FunWebProducts; InfoPath.2; .NET CLR 2.0.50727; OfficeLiveConnector.1.3; OfficeLivePatch.0.0): 2, Mozilla/5.0 (iPod; U; CPU iPhone OS 2_2 like Mac OS X; en-us) AppleWebKit/525.18.1 (KHTML, like Gecko) Version/3.1.1 Mobile/5G77a Safari/525.20: 2, Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 5.1; Trident/4.0; YPC 3.2.0; .NET CLR 1.1.4322; .NET CLR 2.0.50727; .NET CLR 3.0.04506.30; .NET CLR 3.0.04506.648; InfoPath.2): 1 } ----------------- #2 by_ua_created_at ---------------- MAP: function(doc) { emit([doc['user_agent'], doc['created_at']], null); } EXAMPLE OUTPUT (Key, Value) ["8900a/1.2 Mozilla/4.0 (compatible; MSIE 6.0; Windows CE; IEMobile 7.6)", "2009/10/11 13:02:46 +0000"], NULL ---------------- Going through Fulton to view my data, it does not seem it should be 4GB worth but I am missing something. Any insight would be very much appreciated. Thanks, Ryan --Boundary_(ID_dYU4SjAquQEUmNC0lR90Vw)--