Mailing-List: contact user-help@couchdb.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@couchdb.apache.org
Received-SPF: pass (athena.apache.org: domain of omarshall@facilityone.com
 designates 216.135.43.146 as permitted sender)
Message-ID: <4DC18FC7.10908@facilityone.com>
Date: Wed, 04 May 2011 13:41:27 -0400
From: Owen Marshall <omarshall@facilityone.com>
User-Agent: Mozilla/5.0 (X11; U; Linux i686 (x86_64); en-US;
 rv:1.9.2.17) Gecko/20110414 Lightning/1.0b2 Thunderbird/3.1.10
MIME-Version: 1.0
To: user@couchdb.apache.org
CC: jim.klo@sri.com, "Poyau, John" <john.poyau@lmco.com>
Subject: Re: Document Timestamp On Replication
References: <9946CE35511F1449ADF88A3F647DFE2816B0602907@HVXMSP8.us.lmco.com>
 <4DC067DD.5070508@facilityone.com>
 <9946CE35511F1449ADF88A3F647DFE2816B06F1724@HVXMSP8.us.lmco.com>
 <4DC163B2.10502@facilityone.com>
 <9946CE35511F1449ADF88A3F647DFE2816B06F1FA1@HVXMSP8.us.lmco.com>
 <4DC17B73.3010407@facilityone.com>
 <2C4218CE-6352-4F06-9F23-38E77CFFB254@sri.com>
In-Reply-To: <2C4218CE-6352-4F06-9F23-38E77CFFB254@sri.com>
Content-Type: multipart/signed; micalg=pgp-sha1;
 protocol="application/pgp-signature";
 boundary="------------enigB34DA5A6883CA4A55ED61680"

--------------enigB34DA5A6883CA4A55ED61680
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable

On 05/04/2011 12:36 PM, Jim Klo wrote:
> We need to be able to paginate through that range and be guaranteed tha=
t it's not going to be modified via some update happening in another thre=
ad.
>=20
>[...] I request a range of objects at 12:00pm... I need to be able to pa=
ginate through that range probably until 12:05pm, potentially, without an=
y updates between 12:00 and 12:05 effecting the result set.

Ah, *now* I see what you are going for.

First, pretend we are working only with one node. This is important :)

Let's assume that we are only concerned about filtering out new writes
-- that is, we want to be able to run a query and page around from 12:00
- 12:05, and not see a document added at 12:01.

So, include the document update time inside the doc, make that part (or
all!) of your view's key, and use endkey to ensure that you only see
documents before 12:00.

Handling updates becomes a bit more complex, but it's all based on how
your application needs to work.

One idea would be to, on update, store the old document inline. So you'd
go from:

{id: A, updatetime: 12:00, foo:bar} -> {id: A, updatetime: 12:01, foo:
baz, history: {id: A, updatetime: 12:00, foo: bar}}

Then your view just emits all updatetimes for documents *AND* the
updatetimes for all history values. You can use the same endkey filter
as before.

(That's assuming this level of tracking is needed for your program. It
may not be -- but that's up to you. And you could also decouple history
from individual documents. There are plenty of ways to skin this
particular cat.)

OK, now... this works fine for one node -- why won't it work for 2 or mor=
e?

Remember that replication is _not special_; the act of replicating a
document from n1->n2 is equivalent to PUTting/POSTing that document on
n1, then on n2.

So, if a user runs this view on N1 at 12:00, and changes/adds are
replicated in from N2 at 12:01, **it doesn't matter!** Those documents
will have an updatetime > 12:00, so they won't be seen.

Now, there are some possible issues that can come up with conflicts, but
you've got to handle those anyway if you want to use replication -- and
keeping track of when a document was replicated in won't help you with
that. As a matter of fact, that will have more pain points, compared to
well-understood conflicts.

Replication is just another insert/update stream. If you write your view
to work properly on one node (in your case, show only updates before a
given time), it's going to work when you replicate with other nodes.

See:
http://wiki.apache.org/couchdb/HTTP_view_API
http://wiki.apache.org/couchdb/View_collation

for more.

--=20
Owen Marshall
FacilityONE
omarshall@facilityone.com | (502) 805-2126


--------------enigB34DA5A6883CA4A55ED61680
Content-Type: application/pgp-signature; name="signature.asc"
Content-Description: OpenPGP digital signature
Content-Disposition: attachment; filename="signature.asc"

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.10 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iEYEARECAAYFAk3Bj8kACgkQcoY1cxL6GH25IACgtnqq0p0EJvueCkotaGjDTknh
bLIAn2LyK7HbWIKVaPpad8+nVg7r8Yb6
=Qx4k
-----END PGP SIGNATURE-----

--------------enigB34DA5A6883CA4A55ED61680--