Mailing-List: contact dev-help@couchdb.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@couchdb.apache.org
Received-SPF: pass (nike.apache.org: domain of paul.joseph.davis@gmail.com
 designates 209.85.212.52 as permitted sender)
MIME-Version: 1.0
In-Reply-To: <AFEDE10A-598F-42A9-860B-3C165A7A9143@apache.org>
References: 
 <2073127356.24602.1324243590757.JavaMail.tomcat@hel.zones.apache.org>
 <1765975633.44958.1324877790680.JavaMail.tomcat@hel.zones.apache.org>
 <CAN-3CBJpRQ3ya61YOuObunwa2uxiVAvmQCadMtU5me0KAqUpdQ@mail.gmail.com>
 <F740EBE2-020D-4B18-A126-D34C8AEDECCD@dionne-associates.com>
 <CAN-3CB+U3a8=sfE-PZo6pCpO+por7Dt_jZQasNbpCY+xU1-cqg@mail.gmail.com>
 <CAAL6JQj4g761nJ829Xj_pz_pUpM5r0KwQWBPhJMKywPvoHBO7g@mail.gmail.com>
 <CAJNb-9ooAjh1aC=tWgfcXYfdgZd1haxceeKTOtnDBXiZtGUX7A@mail.gmail.com>
 <CAAL6JQi+6oau5ozHKYYnYt5ATVfeivF4SG3W4htqGUgVF+Yt0g@mail.gmail.com>
 <CAN-3CBLYqh6CTF2x=Z66CshKm2FSofVHpAhz0akoW4SVz4kHyQ@mail.gmail.com>
 <CAAL6JQiUVVdpRuDrsUHQOBThO=rXQ6qiEDr_2hpdg-7RvSODnA@mail.gmail.com>
 <CAN-3CBKRBoHGi11VqY+9HyXJ8O=ANEeqo8hm4wJVwSn3b4DXcw@mail.gmail.com>
 <CAAL6JQg_OOc2-EruKc-AXsA3C72d-wrMb28+TA83FYLXDF5jBA@mail.gmail.com>
 <CAN-3CBKmZ4PYRMD7RfNe8qOv7uohRoCRehQWK-2xv_8yZBitWw@mail.gmail.com>
 <AFEDE10A-598F-42A9-860B-3C165A7A9143@apache.org>
From: Paul Davis <paul.joseph.davis@gmail.com>
Date: Wed, 28 Dec 2011 19:35:20 -0600
Message-ID: 
 <CAJ_m3YDcdno7Rk4_qZNdmUdsgNir1apNoTXD=vQX4XMWXk25yA@mail.gmail.com>
Subject: Re: [jira] [Commented] (COUCHDB-1367) When settings revs_limit on db
 - the db increases its update_seq counter when viewing stats - but not when
 getting changes
To: dev@couchdb.apache.org
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable

On Wed, Dec 28, 2011 at 7:02 PM, Damien Katz <damien@apache.org> wrote:
>
> On Dec 28, 2011, at 4:21 PM, Jason Smith wrote:
>
>> On Thu, Dec 29, 2011 at 1:44 AM, Randall Leeds <randall.leeds@gmail.com>=
 wrote:
>>> On Tue, Dec 27, 2011 at 22:37, Jason Smith <jhs@iriscouch.com> wrote:
>>>> On Wed, Dec 28, 2011 at 9:38 AM, Randall Leeds <randall.leeds@gmail.co=
m> wrote:
>>>>> On Tue, Dec 27, 2011 at 05:22, Jason Smith <jhs@iriscouch.com> wrote:
>>>>>> On Tue, Dec 27, 2011 at 5:04 AM, Randall Leeds <randall.leeds@gmail.=
com> wrote:
>>>> Your idea improves consistency and orthogonality. It also solves the
>>>> problem of how to enumerate _local docs. (AFAIK there is no way to
>>>> list them all, not via _all_docs, or _changes, or a view).
>>>>
>>>> But it doesn't solve the larger problem: How to follow a _changes feed
>>>> and know when you have caught up. Both Bob N. and I independently did
>>>> the following for our projects:
>>>>
>>>> 1. GET /db and wrongly assume update_seq will appear in the changes fe=
ed
>>>> 2. GET /db/_changes?feed=3Dcontinuous
>>>> 3. Break when a change has .seq >=3D update_seq
>>>>
>>>> Suppose you have step 0: Update _security or _revs_limit. The loop
>>>> will never break.
>>>>
>>>> You propose (WLOG) _changes?comprehensive=3Dtrue which guarantees a
>>>> change equal or greater than update_seq. That's cool, but IMO app
>>>> developers now have to add code to ignore irrelevant changes like
>>>> those containing replication checkpoints.
>>>
>>> All great points.
>>>
>>>>
>>>> I propose (WLOG) update_sikh in the db header which is the seq id of
>>>> the latest *document* update. App developers modify their step 1 to
>>>> use update_sikh instead of update_seq.
>>>>
>>>> Is that an accurate synopsis?
>>>
>>> Yes. If we decide to go this route I would rather see _revs_limit and
>>> _security stop bumping update_seq because I find it confusing that
>>> update_seq is then not directly related to the by_seq tree.
>>
>> "update_seq is not directly related to the by_seq tree." That is a
>> very powerful observation. My idea has less orthogonality and
>> conceptual simplicity.
>>
>> The only mystery is why update_seq did this in the first place. Did
>> Damien have a reason, from hard-won experience?
>
> I wanted to give an easy way to monitor other changes to the database hea=
der, for things like automated backup and admin tool UIs, though I don't kn=
ow if anything uses it. It was easiest to do it this way, and I ddn't think=
 about the problems it might cause with users expecting it to correspond wi=
th the by_seq index. With all the confusion it's caused, I'd be fine with t=
aking it out. Or maybe put in a db version_seq header, that's like a change=
s feed but for all types of updates to the db.
>

Having a _changes and _events feed I think makes good sense. It could
get us away from the db update notifier stuff as well if we
consolidated things in there.

Though as a note we need to remember to fix the triggering mechanism
for _view updates because of _purge. I think the simplest way would be
to just check both the _update and _purge seq's instead of just
relying on _purge to have changed the _update seq. Luckily we store
both of these thigns already so it should just be a matter of updating
the logic in the guards when grabbing a view group.

>>
>>> The one exception raised so far is your observation that
>>> it may be useful for doing backups. However, if that's the only real
>>> exception, perhaps we should surface some other thing more directly
>>> analogous to the UNIX crime.
>>
>> Meh, the backups argument is okay, but weak. If you depend on it,
>> you'll miss _local docs (replication checkpoints, so that's no small
>> error). You still have to back up .ini files, and perhaps logs and
>> view files. So my suggestion for Unix crime is just that: OS
>> timestamps.
>>
>> --
>> Iris Couch
>