jackrabbit-oak-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From chet...@apache.org
Subject svn commit: r1493053 - in /jackrabbit/oak/trunk/oak-mongomk: README.md src/main/java/org/apache/jackrabbit/mongomk/Commit.java
Date Fri, 14 Jun 2013 12:46:40 GMT
Author: chetanm
Date: Fri Jun 14 12:46:39 2013
New Revision: 1493053

URL: http://svn.apache.org/r1493053
Log:
OAK-619 Lock-free MongoMK implementation

Documentation Work
-- Updated README with details around background job and some open topics
-- Changed variable names for better readability of code

Modified:
    jackrabbit/oak/trunk/oak-mongomk/README.md
    jackrabbit/oak/trunk/oak-mongomk/src/main/java/org/apache/jackrabbit/mongomk/Commit.java

Modified: jackrabbit/oak/trunk/oak-mongomk/README.md
URL: http://svn.apache.org/viewvc/jackrabbit/oak/trunk/oak-mongomk/README.md?rev=1493053&r1=1493052&r2=1493053&view=diff
==============================================================================
--- jackrabbit/oak/trunk/oak-mongomk/README.md (original)
+++ jackrabbit/oak/trunk/oak-mongomk/README.md Fri Jun 14 12:46:39 2013
@@ -1,139 +1,167 @@
-Oak MongoMK
-===========
-
-This module contains a `MicroKernel` implementation using MongoDB to persist
-content.
-
-Content Model
--------------
-
-The `MongoMK` stores each node in a separate MongoDB document and updates to
-a node are stored by adding new revision/value pairs to the document. This way
-the previous state of a node is preserved and can still be retrieved by a
-session looking a given snapshot (revision) of the repository.
-
-The basic MongoDB document of a node in Oak looks like this:
-
-    {
-        "_id" : "1:/node",
-        "_deleted" : {
-            "r13f3875b5d1-0-1" : "false"
-        },
-        "_lastRev" : {
-            "1" : "r13f3875b5d1-0-1"
-        },
-        "_modified" : NumberLong(274208361),
-        "_revisions" : {
-            "r13f3875b5d1-0-1" : "true"
-        }
-    }
-
-All fields in the above document are metadata and are not exposed through the
-Oak API.
-
-The `_id` field is used as the primary key and consists of a combination of the
-depth of the path and the path itself. This is an optimization to align sibling
-keys in the index.
-
-The `_deleted` sub-document contains the revision this node was created in. In
-the above example the root node was created in revision `r13f3875b5d1-0-1`. If
-the node is later deleted, the `_deleted` sub-document will get a new field with
-the revision the node was deleted in.
-
-The sub-document `_lastRev` contains the last revision written to this node by
-each cluster node. In the above example the MongoMK cluster node with id `1`
-modified the node the last time in revision `r13f3875b5d1-0-1`, when it created
-the node.
-
-The `_modified` field contains a low-resolution timestamp when the node was last
-modified. The time resolution is five seconds.
-
-Finally, the `_revision` sub-document contains commit information about changes
-marked with a revision. E.g. the single entry in the above document tells us
-that everything marked with revision `r13f3875b5d1-0-1` is committed and
-therefore valid.
-
-Adding a property `prop` with value `foo` to the node in a next step will
-result in the following document:
-
-    {
-        "_deleted" : {
-            "r13f3875b5d1-0-1" : "false"
-        },
-        "_id" : "1:/node",
-        "_lastRev" : {
-            "1" : "r13f38818ab6-0-1"
-        },
-        "_modified" : NumberLong(274208516),
-        "_revisions" : {
-            "r13f3875b5d1-0-1" : "true",
-            "r13f38818ab6-0-1" : "true"
-        },
-        "prop" : {
-            "r13f38818ab6-0-1" : "\"foo\""
-        }
-    }
-
-Now the document contains a new sub-document with the name of the new property.
-The value of the property is annotated with the revision the property was set.
-With each successful commit to this node, a new field is added to the
-`_revision` sub-document. Similarly the `_lastRev` sub-document and `_modified`
-field are updated.
-
-After the node is deleted the document looks like this:
-
-    {
-        "_deleted" : {
-            "r13f3875b5d1-0-1" : "false",
-            "r13f38835063-2-1" : "true"
-        },
-        "_id" : "1:/node",
-        "_lastRev" : {
-            "1" : "r13f38835063-2-1"
-        },
-        "_modified" : NumberLong(274208539),
-        "_revisions" : {
-            "r13f3875b5d1-0-1" : "true",
-            "r13f38818ab6-0-1" : "true",
-            "r13f38835063-2-1" : "true"
-        },
-        "prop" : {
-            "r13f38818ab6-0-1" : "\"foo\""
-        }
-    }
-
-The `_deleted` sub-document now contains a `r13f38835063-2-1` field marking the
-node as deleted in this revision.
-
-Reading the node in previous revisions is still possible, even if it is now
-marked as deleted as of revision `r13f38835063-2-1`.
-
-Revision Model
---------------
-
-* Explain revision, cluster node id, etc.
-* Explain branches
-
-
-
-License
--------
-
-(see the top-level [LICENSE.txt](../LICENSE.txt) for full license details)
-
-Collective work: Copyright 2013 The Apache Software Foundation.
-
-Licensed to the Apache Software Foundation (ASF) under one or more
-contributor license agreements.  See the NOTICE file distributed with
-this work for additional information regarding copyright ownership.
-The ASF licenses this file to You under the Apache License, Version 2.0
-(the "License"); you may not use this file except in compliance with
-the License.  You may obtain a copy of the License at
-
-     http://www.apache.org/licenses/LICENSE-2.0
-
-Unless required by applicable law or agreed to in writing, software
-distributed under the License is distributed on an "AS IS" BASIS,
-WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-See the License for the specific language governing permissions and
-limitations under the License.
+Oak MongoMK
+===========
+
+This module contains a `MicroKernel` implementation using MongoDB to persist
+content.
+
+Content Model
+-------------
+
+The `MongoMK` stores each node in a separate MongoDB document and updates to
+a node are stored by adding new revision/value pairs to the document. This way
+the previous state of a node is preserved and can still be retrieved by a
+session looking a given snapshot (revision) of the repository.
+
+The basic MongoDB document of a node in Oak looks like this:
+
+    {
+        "_id" : "1:/node",
+        "_deleted" : {
+            "r13f3875b5d1-0-1" : "false"
+        },
+        "_lastRev" : {
+            "1" : "r13f3875b5d1-0-1"
+        },
+        "_commitRoot": {
+            "r13f3875b5d1-0-1": 0
+        },
+        "_modified" : NumberLong(274208361),
+        "_revisions" : {
+            "r13f3875b5d1-0-1" : "true"
+        }
+    }
+
+All fields in the above document are metadata and are not exposed through the
+Oak API.
+
+The `_id` field is used as the primary key and consists of a combination of the
+depth of the path and the path itself. This is an optimization to align sibling
+keys in the index.
+
+The `_deleted` sub-document contains the revision this node was created in. In
+the above example the root node was created in revision `r13f3875b5d1-0-1`. If
+the node is later deleted, the `_deleted` sub-document will get a new field with
+the revision the node was deleted in.
+
+The sub-document `_lastRev` contains the last revision written to this node by
+each cluster node. In the above example the MongoMK cluster node with id `1`
+modified the node the last time in revision `r13f3875b5d1-0-1`, when it created
+the node.
+
+The `_modified` field contains a low-resolution timestamp when the node was last
+modified. The time resolution is five seconds.
+
+The sub-document `_commitRoot` contains commit root depth for the commit in which
+the node was created against the revision.
+
+Finally, the `_revision` sub-document contains commit information about changes
+marked with a revision. E.g. the single entry in the above document tells us
+that everything marked with revision `r13f3875b5d1-0-1` is committed and
+therefore valid. In case the change is done in a branch then the value would be the
+base revision. It is only added for thode nodes which happen to be the commit root
+for any give commit.
+
+Adding a property `prop` with value `foo` to the node in a next step will
+result in the following document:
+
+    {
+        "_deleted" : {
+            "r13f3875b5d1-0-1" : "false"
+        },
+        "_id" : "1:/node",
+        "_lastRev" : {
+            "1" : "r13f38818ab6-0-1"
+        },
+        "_modified" : NumberLong(274208516),
+        "_revisions" : {
+            "r13f3875b5d1-0-1" : "true",
+            "r13f38818ab6-0-1" : "true"
+        },
+        "prop" : {
+            "r13f38818ab6-0-1" : "\"foo\""
+        }
+    }
+
+Now the document contains a new sub-document with the name of the new property.
+The value of the property is annotated with the revision the property was set.
+With each successful commit to this node, a new field is added to the
+`_revision` sub-document. Similarly the `_lastRev` sub-document and `_modified`
+field are updated.
+
+After the node is deleted the document looks like this:
+
+    {
+        "_deleted" : {
+            "r13f3875b5d1-0-1" : "false",
+            "r13f38835063-2-1" : "true"
+        },
+        "_id" : "1:/node",
+        "_lastRev" : {
+            "1" : "r13f38835063-2-1"
+        },
+        "_modified" : NumberLong(274208539),
+        "_revisions" : {
+            "r13f3875b5d1-0-1" : "true",
+            "r13f38818ab6-0-1" : "true",
+            "r13f38835063-2-1" : "true"
+        },
+        "prop" : {
+            "r13f38818ab6-0-1" : "\"foo\""
+        }
+    }
+
+The `_deleted` sub-document now contains a `r13f38835063-2-1` field marking the
+node as deleted in this revision.
+
+Reading the node in previous revisions is still possible, even if it is now
+marked as deleted as of revision `r13f38835063-2-1`.
+
+Revision Model
+--------------
+
+* Explain revision, cluster node id, etc.
+* Explain branches
+
+Background Operations
+---------------------
+Each MongoMK instance connecting to same database in Mongo server performs certain backgroun
task
+
+### Renew Cluster Id Lease
+
+### Background Writes
+
+While performing commits there are certain nodes which are modified but do not become part
+of commit. For example when a node under /a/b/c is updated then the `_lastRev` property also
+needs to be updated to the commit revision. Such changes are accumulated and flushed periodically
+through a asynch job
+
+### Background Reads
+
+
+Pending Topics
+--------------
+
+### Conflict Detection and Handling
+
+
+License
+-------
+
+(see the top-level [LICENSE.txt](../LICENSE.txt) for full license details)
+
+Collective work: Copyright 2013 The Apache Software Foundation.
+
+Licensed to the Apache Software Foundation (ASF) under one or more
+contributor license agreements.  See the NOTICE file distributed with
+this work for additional information regarding copyright ownership.
+The ASF licenses this file to You under the Apache License, Version 2.0
+(the "License"); you may not use this file except in compliance with
+the License.  You may obtain a copy of the License at
+
+     http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.

Modified: jackrabbit/oak/trunk/oak-mongomk/src/main/java/org/apache/jackrabbit/mongomk/Commit.java
URL: http://svn.apache.org/viewvc/jackrabbit/oak/trunk/oak-mongomk/src/main/java/org/apache/jackrabbit/mongomk/Commit.java?rev=1493053&r1=1493052&r2=1493053&view=diff
==============================================================================
--- jackrabbit/oak/trunk/oak-mongomk/src/main/java/org/apache/jackrabbit/mongomk/Commit.java
(original)
+++ jackrabbit/oak/trunk/oak-mongomk/src/main/java/org/apache/jackrabbit/mongomk/Commit.java
Fri Jun 14 12:46:39 2013
@@ -60,7 +60,12 @@ public class Commit {
     private final Revision revision;
     private HashMap<String, UpdateOp> operations = new HashMap<String, UpdateOp>();
     private JsopWriter diff = new JsopStream();
-    private HashSet<String> changedNodes = new HashSet<String>();
+
+    /**
+     * List of all node paths which have been modified in this commit. In addition to the
nodes
+     * which are actually changed it also contains there parent node paths
+     */
+    private HashSet<String> modifiedNodes = new HashSet<String>();
     
     private HashSet<String> addedNodes = new HashSet<String>();
     private HashSet<String> removedNodes = new HashSet<String>();
@@ -152,18 +157,18 @@ public class Commit {
     /**
      * Apply the changes to the document store (to update MongoDB).
      *
-     * @param baseRevision the base revision of this commit. Currently only
+     * @param baseBranchRevision the base revision of this commit. Currently only
      *                     used for branch commits.
      */
-    void applyToDocumentStore(Revision baseRevision) {
+    void applyToDocumentStore(Revision baseBranchRevision) {
         // the value in _revisions.<revision> property of the commit root node
         // regular commits use "true", which makes the commit visible to
         // other readers. branch commits use the base revision to indicate
         // the visibility of the commit
-        String commitValue = baseRevision != null ? baseRevision.toString() : "true";
+        String commitValue = baseBranchRevision != null ? baseBranchRevision.toString() :
"true";
         DocumentStore store = mk.getDocumentStore();
         String commitRootPath = null;
-        if (baseRevision != null) {
+        if (baseBranchRevision != null) {
             // branch commits always use root node as commit root
             commitRootPath = "/";
         }
@@ -172,6 +177,8 @@ public class Commit {
         // operations are added to this list before they are executed,
         // so that all operations can be rolled back if there is a conflict
         ArrayList<UpdateOp> opLog = new ArrayList<UpdateOp>();
+
+        //Compute the commit root
         for (String p : operations.keySet()) {
             markChanged(p);
             if (commitRootPath == null) {
@@ -472,7 +479,7 @@ public class Commit {
             }
             list.add(p);
         }
-        for (String path : changedNodes) {
+        for (String path : modifiedNodes) {
             ArrayList<String> added = new ArrayList<String>();
             ArrayList<String> removed = new ArrayList<String>();
             ArrayList<String> changed = nodesWithChangedChildren.get(path);
@@ -508,7 +515,7 @@ public class Commit {
             throw new IllegalArgumentException("path: " + path);
         }
         while (true) {
-            changedNodes.add(path);
+            modifiedNodes.add(path);
             if (PathUtils.denotesRoot(path)) {
                 break;
             }



Mime
View raw message