couchdb-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Paul Davis <paul.joseph.da...@gmail.com>
Subject Re: [jira] [Commented] (COUCHDB-1367) When settings revs_limit on db - the db increases its update_seq counter when viewing stats - but not when getting changes
Date Thu, 29 Dec 2011 01:35:20 GMT
On Wed, Dec 28, 2011 at 7:02 PM, Damien Katz <damien@apache.org> wrote:
>
> On Dec 28, 2011, at 4:21 PM, Jason Smith wrote:
>
>> On Thu, Dec 29, 2011 at 1:44 AM, Randall Leeds <randall.leeds@gmail.com> wrote:
>>> On Tue, Dec 27, 2011 at 22:37, Jason Smith <jhs@iriscouch.com> wrote:
>>>> On Wed, Dec 28, 2011 at 9:38 AM, Randall Leeds <randall.leeds@gmail.com>
wrote:
>>>>> On Tue, Dec 27, 2011 at 05:22, Jason Smith <jhs@iriscouch.com>
wrote:
>>>>>> On Tue, Dec 27, 2011 at 5:04 AM, Randall Leeds <randall.leeds@gmail.com>
wrote:
>>>> Your idea improves consistency and orthogonality. It also solves the
>>>> problem of how to enumerate _local docs. (AFAIK there is no way to
>>>> list them all, not via _all_docs, or _changes, or a view).
>>>>
>>>> But it doesn't solve the larger problem: How to follow a _changes feed
>>>> and know when you have caught up. Both Bob N. and I independently did
>>>> the following for our projects:
>>>>
>>>> 1. GET /db and wrongly assume update_seq will appear in the changes feed
>>>> 2. GET /db/_changes?feed=continuous
>>>> 3. Break when a change has .seq >= update_seq
>>>>
>>>> Suppose you have step 0: Update _security or _revs_limit. The loop
>>>> will never break.
>>>>
>>>> You propose (WLOG) _changes?comprehensive=true which guarantees a
>>>> change equal or greater than update_seq. That's cool, but IMO app
>>>> developers now have to add code to ignore irrelevant changes like
>>>> those containing replication checkpoints.
>>>
>>> All great points.
>>>
>>>>
>>>> I propose (WLOG) update_sikh in the db header which is the seq id of
>>>> the latest *document* update. App developers modify their step 1 to
>>>> use update_sikh instead of update_seq.
>>>>
>>>> Is that an accurate synopsis?
>>>
>>> Yes. If we decide to go this route I would rather see _revs_limit and
>>> _security stop bumping update_seq because I find it confusing that
>>> update_seq is then not directly related to the by_seq tree.
>>
>> "update_seq is not directly related to the by_seq tree." That is a
>> very powerful observation. My idea has less orthogonality and
>> conceptual simplicity.
>>
>> The only mystery is why update_seq did this in the first place. Did
>> Damien have a reason, from hard-won experience?
>
> I wanted to give an easy way to monitor other changes to the database header, for things
like automated backup and admin tool UIs, though I don't know if anything uses it. It was
easiest to do it this way, and I ddn't think about the problems it might cause with users
expecting it to correspond with the by_seq index. With all the confusion it's caused, I'd
be fine with taking it out. Or maybe put in a db version_seq header, that's like a changes
feed but for all types of updates to the db.
>

Having a _changes and _events feed I think makes good sense. It could
get us away from the db update notifier stuff as well if we
consolidated things in there.

Though as a note we need to remember to fix the triggering mechanism
for _view updates because of _purge. I think the simplest way would be
to just check both the _update and _purge seq's instead of just
relying on _purge to have changed the _update seq. Luckily we store
both of these thigns already so it should just be a matter of updating
the logic in the guards when grabbing a view group.

>>
>>> The one exception raised so far is your observation that
>>> it may be useful for doing backups. However, if that's the only real
>>> exception, perhaps we should surface some other thing more directly
>>> analogous to the UNIX crime.
>>
>> Meh, the backups argument is okay, but weak. If you depend on it,
>> you'll miss _local docs (replication checkpoints, so that's no small
>> error). You still have to back up .ini files, and perhaps logs and
>> view files. So my suggestion for Unix crime is just that: OS
>> timestamps.
>>
>> --
>> Iris Couch
>

Mime
View raw message