Mailing-List: contact user-help@couchdb.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@couchdb.apache.org
Received-SPF: pass (athena.apache.org: domain of bchesneau@gmail.com
 designates 209.85.214.52 as permitted sender)
DomainKey-Signature: a=rsa-sha1; c=nofws;
        d=gmail.com; s=gamma;
        h=mime-version:in-reply-to:references:date:message-id:subject:from:to
         :content-type:content-transfer-encoding;
        b=eaFcJN0LdvVbe9qvmwcrfPg1/VJ0do/MRcDr9Q3DZWDBTiqPgoV9Ahpcu9AyT8VDBm
         3br46cWMU55mu8F4IT1GFhYJWuSILsLlIMcRwAollBeP8W12ZHOxa6v+z2DMIZuXXWlO
         I5E000BR5HtDXrZLvUj9tYx/4eyIc92m0VTuE=
MIME-Version: 1.0
In-Reply-To: <BANLkTinEk7yz0CM+NVWdDS5knmr=bvSx0g@mail.gmail.com>
References: <BANLkTi=NAJB2foEqc-WTKTu8NoeAXq7EXA@mail.gmail.com>
	<BANLkTinv0c0U=Ji0Vz3hPJBThxrug6BTBA@mail.gmail.com>
	<BANLkTinEk7yz0CM+NVWdDS5knmr=bvSx0g@mail.gmail.com>
Date: Mon, 23 May 2011 08:44:23 +0200
Message-ID: <BANLkTikrH76f+0PttHwY-wSy7rdez=-WQw@mail.gmail.com>
Subject: Re: DB that mirrors a view
From: Benoit Chesneau <bchesneau@gmail.com>
To: user@couchdb.apache.org
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable

On Mon, May 23, 2011 at 7:55 AM, afters <afters.mail@gmail.com> wrote:
> Jason,
>
> Thanks for the detailed answer! I've actually written code along the same
> lines as yours (only with eval instead of 'new Function' - thanks for the
> tip).
>
> I'm actually interested in the reduce results, rather than the map result=
s.
> It's still possible to deduce the affected reduce-keys for each change, j=
ust
> by using the map function: just clip the emitted map key according to the
> desired group_level. Alas, it's not possible to deduce the query results =
-
> as reduce depends also on previous changes.
>
> If only I could prevent the view from being updated for a little while, I
> could read the changes feed till I'm in line with the view's seqnum...
>
> Benoit, if you're reading this I'd love to hear more about your fork.
>
> =A0a.
>
> On 23 May 2011 04:25, Jason Smith <jhs@iriscouch.com> wrote:
>
>> I think Benoit has a fork of CouchDB to do this. Also I believe this
>> feature (a _changes filter that only shows docs that change a view) is
>> possibly coming in the future.
>>
>> As a side-note, view queries support an ?update_seq=3Dtrue parameter,
>> which will tell you the seqnum of the database which that view
>> represents.
>>
>> Anyway, probably the best bang-for-buck with NodeJS is simply to
>> download the design document and run the map() functions yourself.
>> They are purely functional, and they have no (or perhaps well-known)
>> side-effects.
>>
>> The basic idea is:
>>
>> 1. Fetch the design document as JSON
>> 2. Extract the .views.whatever.map value as a string
>> 3. Build a wrapper map() function that has a custom emit()
>> 4. Run it for every _changes update
>>
>> Below is my basic idea. I think I've seen it elsewhere, perhaps in
>> Benoit's fork. But this is with Node. It could use some optimization
>> too (don't re-define map() every function call).
>>
>> var ddoc =3D get_ddoc();
>> var couch_map =3D ddoc.views.someview.map;
>>
>> var changed_keys =3D [];
>> function emit(key, val) {
>> =A0console.log("Emit! " + JSON.stringify(key));
>> =A0changed_keys.push(key);
>> }
>>
>> var map =3D new Function("doc, emit", "var couch_map =3D " + couch_map +
>> "; couch_map(doc););
>>
>> changes.forEach(function(change) {
>> =A0map(change.doc, emit);
>> })
>>
>> console.log("These keys were emitted: " + JSON.stringify(changed_keys));
>>
>> > Hi guys,
>> >
>> > I'm giving a serious look into building a db that mirrors the contents=
 of
>> a
>> > view (for running map on the results of map-reduce).
>> >
>> > The hurdle that I'm facing is tracking changes in the view:
>> > It's possible to listen to the db's changes feed and deduce which
>> view-keys
>> > should be updated for a particular seqnum, then query the view with th=
ese
>> > keys. Unfortunately, the view would only return the latest results, th=
at
>> may
>> > easily incorporate changes from later seqnums, thus making my mirrorin=
g
>> db
>> > inconsistent with the view.
>> >
>> > Any thoughts? Has anyone else meddled with turning a view to a db?
>> > It would be useful if I could somehow freeze the update of the view (I
>> guess
>> > I could do it by replicating its db only to a certain point, but that
>> would
>> > mean a whole other db)
>> >
>> > Thanks,
>> > =A0a.
>> >
>>
>>
>>
>> --
>> Iris Couch
>>
>
Hi,

Indeed, in trunk and 1.1 you can pass a _view filter to change then a
view param view=3Ddname/vname that allows you to get all changes using
the map function and then know if your view have been updated. Maybe
that can help.

- beno=EEt