jackrabbit-oak-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Stefan Egli (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (OAK-6276) expose way to detect "eventual consistency delay"
Date Thu, 01 Jun 2017 07:50:04 GMT

    [ https://issues.apache.org/jira/browse/OAK-6276?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16032599#comment-16032599
] 

Stefan Egli commented on OAK-6276:
----------------------------------

bq. So I would treat just like current support for Clusterable which is currently only implemented
by DocumentNodeStore.
My point about a SegmentNodeStore implementation for this was just that it might be good if
we had a broad idea of how this could be done (easily), not that we're actually doing it.
Agree that it's only needed for the cluster case anyway. But perhaps some time in the future
there might be a clustered SegmentNodeStore - at which point this could become relevant again.
bq. what happens if the token never occurs?
bq. one node might go through r1 -> r2 -> r4, another node might go through r1 ->
r3 -> r4
as long as there's a way to say that r3 or r4 contains r2 we should be fine. If not then we'd
have an issue.

> expose way to detect "eventual consistency delay"
> -------------------------------------------------
>
>                 Key: OAK-6276
>                 URL: https://issues.apache.org/jira/browse/OAK-6276
>             Project: Jackrabbit Oak
>          Issue Type: New Feature
>          Components: api
>            Reporter: Stefan Egli
>
> I have a requirement to support an external messaging channel (eg Kafka) between Oak-based
instances of the same cluster. As part of handling those messages the target instance in some
cases might have to access data from the repository. 
> Now with DocumentNodeStore's eventual consistency that data might not 'travel' from the
source to the target instance as fast as is the case for the external message.
> Therefore the need arises to be able to delay such messages (on the target instance)
until the repository sees (at least) the data the source instance wrote when sending off the
message.
> This ticket is to equip Oak with any feasible way for higher level code to generally
speaking detect such an "eventual consistency delay".
> One simple idea that comes to mind is to expose the current _head revision vector_ (or
that from a particular session, but that might not be required, ie be too complicated). The
source instance could get the local head revision vector, piggyback that on the message, then
that could be compared on the target instance with its head state. If that turns out to be
older, then it could do a wait and retry. (Nicer would of course be if there would even be
a call-back - but in theory that could also be implemented ontop of an Observer).
> One means to expose the head revision vector would be via a repository descriptor (which
on access returns the current value, similar to [how discovery-lite does it|https://github.com/apache/jackrabbit-oak/blob/2634dbde9aedc2549f0512285e9abee5858b256f/oak-core/src/main/java/org/apache/jackrabbit/oak/plugins/document/DocumentDiscoveryLiteService.java#L246]).
And the format could be normalized as eg {{longs}} (eg {{\[1496071927014, 1496071926243]}}
(instead of {{\["r15c54d532ec-0-1", "r15c54d532ec-0-2"]}} to avoid leaking the revision format
explicitly).



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

Mime
View raw message