Mailing-List: contact user-help@couchdb.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@couchdb.apache.org
Received-SPF: pass (nike.apache.org: domain of paul.joseph.davis@gmail.com
 designates 209.85.211.186 as permitted sender)
DomainKey-Signature: a=rsa-sha1; c=nofws;
        d=gmail.com; s=gamma;
        h=mime-version:in-reply-to:references:from:date:message-id:subject:to
         :content-type:content-transfer-encoding;
        b=RKRtMYaPBtTVK37eUa8f2r8A0eV2TdG/dey1wMr0pQHgOLbgi1AFGYwBXjFZHar1o+
         mTC0QlXVYfShNx82YbTXyCP1CcdGqWgLnOYyLzxVPqaSqlDOL5j0DLQ79VbTNRbzheUQ
         UOxQCHws70Z/SG4Syqac2zAMTiJSa8A263sI8=
MIME-Version: 1.0
In-Reply-To: 
 <CE1513D67312104D86366A5A25E2B1EC27674442DF@EXVMBX017-12.exch017.msoutlookonline.net>
References: <F49C2E88-AB7B-4ED1-8B9C-8BD73F2F2C23@mac.com>
	<CE1513D67312104D86366A5A25E2B1EC27674442DF@EXVMBX017-12.exch017.msoutlookonline.net>
From: Paul Davis <paul.joseph.davis@gmail.com>
Date: Mon, 26 Oct 2009 16:38:29 -0400
Message-ID: <e2111bbb0910261338k3974ff2dp2181106a81180dee@mail.gmail.com>
Subject: Re: Size of view file
To: user@couchdb.apache.org
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable

Ryan,

Kevin's got it right here. Try his view to see what size you get. My
guess is that UA's are different enough that you would've be getting
reduciness errors on newer versions.

Paul Davis

This might be helpful:

http://mail-archives.apache.org/mod_mbox/couchdb-user/200904.mbox/%3Ce2111b=
bb0904090817j7ba10b34pe53ce2fd3c5c1590@mail.gmail.com%3E

On Mon, Oct 26, 2009 at 4:29 PM, Kevin Ferguson <kevin@meebo-inc.com> wrote=
:
> For #1, have you considered a view like:
>
> MAP:
> function(doc) {
> =A0 =A0 =A0 =A0 datetime =3D doc.created_at;
> =A0 =A0 =A0 =A0 year =3D parseInt(datetime.substr(0, 4));
> =A0 =A0 =A0 =A0 month =3D parseInt(datetime.substr(5, 2), 10);
> =A0 =A0 =A0 =A0 day =3D parseInt(datetime.substr(8, 2), 10);
> =A0 =A0 =A0 =A0 emit([year, month, day, doc.user_agent], val );
> =A0 =A0 =A0 }
> REDUCE:
> function(k,v,r) { return sum(v); }
>
> Then you can query with startkey=3D[y,m,d], endkey=3D[y,m,d,{}], group=3D=
true and get the count for each user-agent on that day. =A0I think the outp=
ut will be smaller too, but I don't know a whole lot about the view engine =
internals.
>
> Kevin
>
> ________________________________________
> From: Ryan Richins [richinsr@mac.com]
> Sent: Monday, October 26, 2009 1:24 PM
> To: user@couchdb.apache.org
> Subject: Size of view file
>
> I am working on a project where i have 12k documents and the size of
> the db is 11MB but the view file is is over 4GB. =A0Obviously I am doing
> something wrong with my views to make the file so large. =A0I was hoping
> to get some input as to where my problem might be.
>
> Running couchdb 0.90
>
> Each document has 3 attributes one of which is 'User Agent'. =A0For each
> attribute I have the following views defined
> "by_<attribute>_total_date" and "by_<attribute> _created_at". =A0Below
> is the code for the 2 views that deal with User Agent. =A0The same code
> is used to define the views for the other 2 attributes except
> doc.user_agent is replace by doc.<attribute>
>
> My guess is the problem lies somewhere in the
> "by_<attribute>_total_date" since every other view I have returns NULL
> for the value.
>
> #1 by_ua_total_date
> -----------------
> MAP:
> function(doc) {
> =A0 =A0 =A0 =A0 var val =3D {};
> =A0 =A0 =A0 =A0 datetime =3D doc.created_at;
> =A0 =A0 =A0 =A0 year =3D parseInt(datetime.substr(0, 4));
> =A0 =A0 =A0 =A0 month =3D parseInt(datetime.substr(5, 2), 10);
> =A0 =A0 =A0 =A0 day =3D parseInt(datetime.substr(8, 2), 10);
> =A0 =A0 =A0 =A0 val[doc.user_agent] =3D 1;
> =A0 =A0 =A0 =A0 emit([year, month, day], val );
> =A0 =A0 =A0 }
>
> REDUCE:
> function (keys, values, rereduce) {
> =A0 =A0 =A0 =A0 var rv =3D {};
> =A0 =A0 =A0 =A0 for (i in values) {
> =A0 =A0 =A0 =A0 =A0 var value =3D values[i];
> =A0 =A0 =A0 =A0 =A0 for (k in value) {
> =A0 =A0 =A0 =A0 =A0 =A0 rv[k] =3D (rv[k] || 0) + value[k];
> =A0 =A0 =A0 =A0 =A0 }
> =A0 =A0 =A0 =A0 }
> =A0 =A0 =A0 =A0 return rv;
> =A0 =A0 =A0 }
>
>
> EXAMPLE OUTPUT (Key, Value)
> [2009, 9, 6], {Mozilla/5.0 (iPod; U; CPU iPhone OS 2_2_1 like Mac OS
> X; en-us) AppleWebKit/525.18.1 (KHTML, like Gecko) Version/3.1.1
> Mobile/5H11a Safari/525.20: 5, Mozilla/4.0 (compatible; MSIE 6.0;
> Windows NT 5.1; SV1; FunWebProducts; InfoPath.2; .NET CLR 2.0.50727;
> OfficeLiveConnector.1.3; OfficeLivePatch.0.0): 2, Mozilla/5.0 (iPod;
> U; CPU iPhone OS 2_2 like Mac OS X; en-us) AppleWebKit/525.18.1
> (KHTML, like Gecko) Version/3.1.1 Mobile/5G77a Safari/525.20: 2,
> Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 5.1; Trident/4.0; YPC
> 3.2.0; .NET CLR 1.1.4322; .NET CLR 2.0.50727; .NET CLR
> 3.0.04506.30; .NET CLR 3.0.04506.648; InfoPath.2): 1 }
> -----------------
>
> #2 by_ua_created_at
> ----------------
> MAP:
> function(doc) {
> =A0 =A0 emit([doc['user_agent'], doc['created_at']], null);
> }
> EXAMPLE OUTPUT (Key, Value)
> ["8900a/1.2 Mozilla/4.0 (compatible; MSIE 6.0; Windows CE; IEMobile
> 7.6)", "2009/10/11 13:02:46 +0000"], NULL
> ----------------
>
>
>
> Going through Fulton to view my data, it does not seem it should be
> 4GB worth but I am missing something. =A0Any insight would be very much
> appreciated.
>
> Thanks,
>
> Ryan
>
>
>
>