Mailing-List: contact user-help@couchdb.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@couchdb.apache.org
Received-SPF: pass (nike.apache.org: domain of jchris@gmail.com designates
 209.85.221.172 as permitted sender)
DomainKey-Signature: a=rsa-sha1; c=nofws;
        d=gmail.com; s=gamma;
        h=content-type:mime-version:subject:from:in-reply-to:date
         :content-transfer-encoding:message-id:references:to:x-mailer;
        b=hGYF+wIdHus904fTNv8g/hLl2XRnuScQYkAgfGjosc5+OSwNE3+6ZWGkaHLmxU9Cs2
         5aGBnKKMcT1o5nx2+1J6lGGqn5dFQYWgv027prpJyJ12jIsyRUfPysXDoQTWQrvc86SY
         1QBNBnpUgWkubboefo8RW0OOMN/PE0/11DRn0=
Content-Type: text/plain; charset=us-ascii
Mime-Version: 1.0 (Apple Message framework v1077)
Subject: Re: Is there anyway to specify a group_level of "id"?
From: J Chris Anderson <jchris@gmail.com>
In-Reply-To: <j2zfa3d0c741004190927j95d8400as513e819da764d97e@mail.gmail.com>
Date: Mon, 19 Apr 2010 09:50:57 -0700
Content-Transfer-Encoding: quoted-printable
Message-Id: <01FC0EB2-79DF-4408-9D13-8E21E7F347F3@gmail.com>
References: <n2kfa3d0c741004172237o89da1f49x1e905a0cc7fcfea8@mail.gmail.com>
 <CDE25EE3-158C-4DCF-8E6C-2B1A40C5B8F0@apache.org>
 <j2zfa3d0c741004190927j95d8400as513e819da764d97e@mail.gmail.com>
To: user@couchdb.apache.org


On Apr 19, 2010, at 9:27 AM, Jarrod Roberson wrote:

> On Mon, Apr 19, 2010 at 10:10 AM, Adam Kocoloski <kocolosk@apache.org> =
wrote:
>> On Apr 18, 2010, at 1:37 AM, Jarrod Roberson wrote:
>>=20
>>=20
>> Hi Jarrod, I'd need a little more detail or an example before I could =
whether what you want to do is possible.  Best,
>>=20
>> Adam
>=20
> I am working on what I think is a clever solution to not being able to
> do variable "select where" sql like selections on CouchDB.
>=20
> here is my map function
>=20
> function(doc)
> {
>  emit(['cnnid', doc.cnnid], null);
>  emit(['guid', doc.guid], null);
>  emit(['src', doc.sourceServer], null);
>  emit(['dest', doc.destServer], null);
> }
>=20
> running a reduce that works with group_level=3D1 it is merging lists =
of
> _ids by field. what the result is a unique list of _ids that match
> each field name when run with keys=3D[['cnnid',"11111111"],["src","a"]].=

> I get output that groups by cnnid and src what I want to do is
> rereduce just the "final" output one more time to reduce the keys down
> to the unique list of _ids from the resulting groups.
>=20

I think this is misuse of CouchDB's reduce, which is really just there =
to provide numerical aggregation (or other constant-space operations). =
I'm surprised you haven't been hit with the reduce_overflow_error yet. =
As your database becomes larger, this will surely happen.

If you want to do something like this, you are better off moving all =
your uniqueness logic to a _list function, and then optimizing your map =
collation to keep the memory usage of your _list low.

Chris

>=20
> curl -X POST -d '{"keys":[["cnnid","82534864"],["src","a"]]}'
> =
http://localhost:5984/transfer_central/_design/transfer/_view/search?group=
=3Dtrue&group_level=3D1
> looks like this
>=20
> {"rows":[
> =
{"key":["cnnid","82534864"],"value":["fdbc746e0026B93BD6FE6f83c80de090","f=
dbc6f930026B92F3075118c8e46f574","fdbc59760026B92F30754e88a5fb1d0a"]},
> =
{"key":["src","a"],"value":["2fe5b7620026B93BD6FE54240135cf78","3028f10100=
26B93BD6FE27430d5ff179","3028f3df0026B93BD6FE1aaf792acaec","48a5d7ab0026B9=
3BD6FE347e40beba22","48a5dada0026B93BD6FE5759f3f61946","48a8630c0026B93BD6=
FE6bc36f72aaf9","673fd21a0026B93BD6FE56b473da6a77","673fd4790026B93BD6FE47=
aeffcaa16b","67af7dbc0026B93BD6FE3134aabda4b6","67af80b60026B93BD6FE132454=
bf17b5"]}
> ]}
>=20
> I tried group_level=3D0 but that does the same thing as 1 in my case. =
I
> tried hacking at the Erlang source to get it to run a another final
> rereduce with a "special" group_level=3D9999 but I didn't have any =
luck
> getting that to work. What I finally resorted to is a List function
> that does the final reduce/merge of the _ids (same thing the reduce
> function is doing really) but I really think it would be clever if I
> could get the reduce to run one last rereduce instead of the List
> function solution.
>=20
> Here is a post about it in more detail
> =
http://www.vertigrated.com/blog/2010/04/where-clauses-like-selects-against=
-couchdb/