Mailing-List: contact dev-help@couchdb.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@couchdb.apache.org
Date: Mon, 22 Aug 2011 19:30:35 +0000 (UTC)
From: "Adam Kocoloski (JIRA)" <jira@apache.org>
To: dev@couchdb.apache.org
Message-ID: 
 <1468132550.2395.1314041435192.JavaMail.tomcat@hel.zones.apache.org>
In-Reply-To: 
 <1136929143.1164.1314022829540.JavaMail.tomcat@hel.zones.apache.org>
Subject: [jira] [Commented] (COUCHDB-1256) Incremental requests to _changes
 can skip revisions
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit


    [ https://issues.apache.org/jira/browse/COUCHDB-1256?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13088952#comment-13088952 ] 

Adam Kocoloski commented on COUCHDB-1256:
-----------------------------------------

Hi Filipe, it is definitely not the case that "if the replicator checkpointed sequence 2, it means it received and processed revision 1-0dc33db52a43872b6f3371cef7de0277 of foo".  Please take a closer look at the full _changes response in the initial report:

curl -Hcontent-type:application/json localhost:5985/revseq/_changes?style=all_docs 
{"results":[ 
{"seq":2,"id":"bar","changes":[{"rev":"1-cc609831f0ca66e8cd3d4c1e0d98108a"}]}, 
{"seq":3,"id":"foo","changes":[{"rev":"1-cc609831f0ca66e8cd3d4c1e0d98108a"},{"rev":"1-0dc33db52a43872b6f3371cef7de0277"}]} 
], 
"last_seq":3}

Revision 1-0dc33d of "foo" shows up in sequence 3, which is after sequence 2.  The replicator should checkpoint sequence 2 iff it has replicated all the entries in the _changes feed up to and including sequence 2.  Once again, that does not include 1-0dc33d.  If I resume the replication from a checkpoint at sequence 2 I will not receive 1-0dc33d.

> Incremental requests to _changes can skip revisions
> ---------------------------------------------------
>
>                 Key: COUCHDB-1256
>                 URL: https://issues.apache.org/jira/browse/COUCHDB-1256
>             Project: CouchDB
>          Issue Type: Bug
>          Components: Replication
>    Affects Versions: 0.10, 0.10.1, 0.10.2, 0.11.1, 0.11.2, 1.0, 1.0.1, 1.0.2, 1.1, 1.0.3
>         Environment: confirmed on Apache CouchDB 1.1.0, bug appears to be present in 1.0.3 and trunk
>            Reporter: Adam Kocoloski
>            Assignee: Adam Kocoloski
>            Priority: Blocker
>             Fix For: 1.0.4, 1.1.1, 1.2
>
>         Attachments: jira-1256-test.diff
>
>
> Requests to _changes with style=all_docs&since=N (requests made by the replicator) are liable to suppress revisions of a document.  The following sequence of curl commands demonstrates the bug:
> curl -X PUT localhost:5985/revseq 
> {"ok":true}
> curl -X PUT -Hcontent-type:application/json localhost:5985/revseq/foo -d '{"a":123}'
> {"ok":true,"id":"foo","rev":"1-0dc33db52a43872b6f3371cef7de0277"}
> curl -X PUT -Hcontent-type:application/json localhost:5985/revseq/bar -d '{"a":456}'
> {"ok":true,"id":"bar","rev":"1-cc609831f0ca66e8cd3d4c1e0d98108a"}
> % stick a conflict revision in foo
> curl -X PUT -Hcontent-type:application/json localhost:5985/revseq/foo?new_edits=false -d '{"_rev":"1-cc609831f0ca66e8cd3d4c1e0d98108a", "a":123}'
> {"ok":true,"id":"foo","rev":"1-cc609831f0ca66e8cd3d4c1e0d98108a"}
> % request without since= gives the expected result
> curl -Hcontent-type:application/json localhost:5985/revseq/_changes?style=all_docs
> {"results":[
> {"seq":2,"id":"bar","changes":[{"rev":"1-cc609831f0ca66e8cd3d4c1e0d98108a"}]},
> {"seq":3,"id":"foo","changes":[{"rev":"1-cc609831f0ca66e8cd3d4c1e0d98108a"},{"rev":"1-0dc33db52a43872b6f3371cef7de0277"}]}
> ],
> "last_seq":3}
> % request starting from since=2 suppresses revision 1-0dc33db52a43872b6f3371cef7de0277 of foo
> macbook:~ (master) $ curl localhost:5985/revseq/_changes?style=all_docs\&since=2
> {"results":[
> {"seq":3,"id":"foo","changes":[{"rev":"1-cc609831f0ca66e8cd3d4c1e0d98108a"}]}
> ],
> "last_seq":3}
> I believe the fix is something like this (though we could refactor further because Style is unused):
> diff --git a/src/couchdb/couch_db.erl b/src/couchdb/couch_db.erl
> index e8705be..65aeca3 100644
> --- a/src/couchdb/couch_db.erl
> +++ b/src/couchdb/couch_db.erl
> @@ -1029,19 +1029,7 @@ changes_since(Db, Style, StartSeq, Fun, Acc) ->
>      changes_since(Db, Style, StartSeq, Fun, [], Acc).
>      
>  changes_since(Db, Style, StartSeq, Fun, Options, Acc) ->
> -    Wrapper = fun(DocInfo, _Offset, Acc2) ->
> -            #doc_info{revs=Revs} = DocInfo,
> -            DocInfo2 =
> -            case Style of
> -            main_only ->
> -                DocInfo;
> -            all_docs ->
> -                % remove revs before the seq
> -                DocInfo#doc_info{revs=[RevInfo ||
> -                    #rev_info{seq=RevSeq}=RevInfo <- Revs, StartSeq < RevSeq]}
> -            end,
> -            Fun(DocInfo2, Acc2)
> -        end,
> +    Wrapper = fun(DocInfo, _Offset, Acc2) -> Fun(DocInfo, Acc2) end,
>      {ok, _LastReduction, AccOut} = couch_btree:fold(by_seq_btree(Db),
>          Wrapper, Acc, [{start_key, StartSeq + 1}] ++ Options),
>      {ok, AccOut}.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira