accumulo-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From ktur...@apache.org
Subject [accumulo-website] branch master updated: Added conditional writer to tour
Date Fri, 08 Dec 2017 22:45:49 GMT
This is an automated email from the ASF dual-hosted git repository.

kturner pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/accumulo-website.git


The following commit(s) were added to refs/heads/master by this push:
     new 52f812c  Added conditional writer to tour
52f812c is described below

commit 52f812cf45d76f84b3c02fcb15708d77cdc2ea37
Author: Keith Turner <kturner@apache.org>
AuthorDate: Thu Dec 7 15:50:22 2017 -0500

    Added conditional writer to tour
---
 _data/tour.yml                  |   2 +
 tour/conditional-writer-code.md |  36 ++++++++++++
 tour/conditional-writer.md      | 127 ++++++++++++++++++++++++++++++++++++++++
 3 files changed, 165 insertions(+)

diff --git a/_data/tour.yml b/_data/tour.yml
index c15d2c1..9bd0287 100644
--- a/_data/tour.yml
+++ b/_data/tour.yml
@@ -8,3 +8,5 @@ docs:
  - ranges-splits
  - batch-scanner
  - batch-scanner-code
+ - conditional-writer
+ - conditional-writer-code
diff --git a/tour/conditional-writer-code.md b/tour/conditional-writer-code.md
new file mode 100644
index 0000000..cd9c0c0
--- /dev/null
+++ b/tour/conditional-writer-code.md
@@ -0,0 +1,36 @@
+---
+title: Conditional Writer Code
+---
+
+Below is a solution to the excercise.
+
+```java
+  static boolean setAddress(Connector conn, String id, String expectedAddr, String newAddr)
{
+    try (ConditionalWriter writer = conn.createConditionalWriter("GothamPD", new ConditionalWriterConfig()))
{
+      Condition condition = new Condition("location", "home");
+      if(expectedAddr != null) {
+        condition.setValue(expectedAddr);
+      }
+      ConditionalMutation mutation = new ConditionalMutation(id, condition);
+      mutation.put("location", "home", newAddr);
+      return writer.write(mutation).getStatus() == Status.ACCEPTED;
+    } catch (Exception e) {
+      throw new RuntimeException(e);
+    }
+  }
+
+```
+
+The following output shows running the example with a conditional writer.
+Threads retry when conditional mutations are rejected.  The final address has
+all three modifications.
+
+```
+Thread  37 attempting change '  1007 Mountain Drive, Gotham, New York  ' -> '  1007 Mountain
Dr, Gotham, New York  '
+Thread  38 attempting change '  1007 Mountain Drive, Gotham, New York  ' -> '1007 Mountain
Drive, Gotham, New York'
+Thread  39 attempting change '  1007 Mountain Drive, Gotham, New York  ' -> '  1007 Mountain
Drive, Gotham, NY  '
+Thread  38 attempting change '  1007 Mountain Dr, Gotham, New York  ' -> '1007 Mountain
Dr, Gotham, New York'
+Thread  39 attempting change '  1007 Mountain Dr, Gotham, New York  ' -> '  1007 Mountain
Dr, Gotham, NY  '
+Thread  39 attempting change '1007 Mountain Dr, Gotham, New York' -> '1007 Mountain Dr,
Gotham, NY'
+Final address : '1007 Mountain Dr, Gotham, NY'
+```
diff --git a/tour/conditional-writer.md b/tour/conditional-writer.md
index 83d468c..dc98d8a 100644
--- a/tour/conditional-writer.md
+++ b/tour/conditional-writer.md
@@ -1,3 +1,130 @@
 ---
 title: Conditional Writer
 ---
+
+Suppose the Gotham PD is storing home addresses for persons of interest in
+Accumulo.  We want to correctly handle the case of multiple users editing the
+same address at the same time. The following sequence of events shows an example
+of how this can go wrong.
+
+ 1. User 0 sets the key `id0001:location:home` to `1007 Mountain Drive, Gotham, New York`
+ 2. User 1 reads `id0001:location:home`
+ 3. User 2 reads `id0001:location:home`
+ 4. User 1 replaces `Drive` with `Dr`
+ 5. User 2 replaces `New York` with `NY`
+ 6. User 1 sets key `id0001:location:home` to `1007 Mountain Dr, Gotham, New York`
+ 7. User 2 sets key `id0001:location:home` to `1007 Mountain Drive, Gotham, NY`
+
+In this situation the changes made by User 1 are lost, ending up with `1007
+Mountain Drive, Gotham, NY` instead of `1007 Mountain Dr, Gotham, NY`.  To
+correctly handle this, Accumulo offers the [ConditionalWriter].  The
+ConditionalWriter atomically checks conditions on a row and only applies a
+mutation when all conditions are satisfied.
+
+## Exercise
+
+The following code simulates the concurrency in the situation above.  The code
+starts multiple threads, with each thread doing the following.
+
+ 1. Read key's value into memory using a scanner
+ 2. Modify the copy in memory.
+ 3. Write out the modified value from memory using a batch writer.
+ 4. If write was unsuccessful, then goto step 1.
+
+This process can result in threads overwriting each other changes.  The problem
+is the batch writer always makes the update, even when the value has
+changed since it was read.
+
+```java
+  static String getAddress(Connector conn, String id) {
+    // The IsolatedScanner ensures partial changes to a row are not seen
+    try (Scanner scanner = new IsolatedScanner(conn.createScanner("GothamPD", Authorizations.EMPTY)))
{
+      scanner.setRange(Range.exact(id, "location", "home"));
+      for (Entry<Key,Value> entry : scanner) {
+        return entry.getValue().toString();
+      }
+      return null;
+    } catch (TableNotFoundException e) {
+      throw new RuntimeException(e);
+    }
+  }
+
+  static boolean setAddress(Connector conn, String id, String expectedAddr, String newAddr)
{
+    try (BatchWriter writer = conn.createBatchWriter("GothamPD", new BatchWriterConfig()))
{
+      Mutation mutation = new Mutation(id);
+      mutation.put("location", "home", newAddr);
+      writer.addMutation(mutation);
+      return true;
+    } catch (Exception e) {
+      throw new RuntimeException(e);
+    }
+  }
+
+  public static Future<Void> modifyAddress(Connector conn, String id, Function<String,String>
modifier) {
+    return CompletableFuture.runAsync(() -> {
+      String currAddr, newAddr;
+      do {
+        currAddr = getAddress(conn, id);
+        newAddr = modifier.apply(currAddr);
+        System.out.printf("Thread %3d attempting change %20s -> %-20s\n",
+            Thread.currentThread().getId(), "'"+currAddr+"'", "'"+newAddr+"'");
+      } while (!setAddress(conn, id, currAddr, newAddr));
+    });
+  }
+
+  static void exercise(MiniAccumuloCluster mac) throws Exception {
+    Connector conn = mac.getConnector("root", "tourguide");
+    conn.tableOperations().create("GothamPD");
+
+    String id = "id0001";
+
+    setAddress(conn, id, null, "  1007 Mountain Drive, Gotham, New York  ");
+
+    // create async operation to trim whitespace
+    Future<Void> future1 = modifyAddress(conn, id, String::trim);
+
+    // create async operation to replace Dr with Drive
+    Future<Void> future2 = modifyAddress(conn, id, addr -> addr.replace("Drive",
"Dr"));
+
+    // create async operation to replace New York with NY
+    Future<Void> future3 = modifyAddress(conn, id, addr -> addr.replace("New York",
"NY"));
+
+    // wait for async operations to complete
+    future1.get();
+    future2.get();
+    future3.get();
+
+    // print the address stored in Accumulo
+    System.out.println("Final address : '"+getAddress(conn, id)+"'");
+  }
+```
+
+The following is one of a few possible outputs.  Notice that only the
+modification of `Drive` to `Dr` shows up in the final output.  The other
+modifications were lost.
+
+```
+Thread  36 attempting change '  1007 Mountain Drive, Gotham, New York  ' -> '1007 Mountain
Drive, Gotham, New York'
+Thread  38 attempting change '  1007 Mountain Drive, Gotham, New York  ' -> '  1007 Mountain
Drive, Gotham, NY  '
+Thread  37 attempting change '  1007 Mountain Drive, Gotham, New York  ' -> '  1007 Mountain
Dr, Gotham, New York  '
+Final address : '  1007 Mountain Dr, Gotham, New York  '
+```
+
+To fix this example, make the following changes in `setAddress()` to use a
+ConditionalWriter.
+
+ * Call [createConditionalWriter] instead of creating a batch writer
+ * Create a [Condition] for the column 'location:home'.  If `expectedAddr` is not null, then
call [setValue] passing `expectedAddr`.  If `expectedAddr` is null, then do nothing else with
the condition. A condition with no value means that column is expected to be absent.
+ * Replace Mutation with a [ConditionalMutation] and pass the condition to its constructor.
+ * Call [write] passing it the conditional mutation.
+ * Return `true` if [getStatus] from the [Result] returned by [write] is [ACCEPTED].
+
+[ConditionalWriter]: {{ site.javadoc_core }}/org/apache/accumulo/core/client/ConditionalWriter.html
+[Result]: {{ site.javadoc_core }}/org/apache/accumulo/core/client/ConditionalWriter.Result.html
+[createConditionalWriter]: {{ site.javadoc_core }}/org/apache/accumulo/core/client/Connector.html#createConditionalWriter(java.lang.String,%20org.apache.accumulo.core.client.ConditionalWriterConfig)
+[Condition]: {{ site.javadoc_core }}/org/apache/accumulo/core/data/Condition.html
+[ConditionalMutation]: {{ site.javadoc_core }}/org/apache/accumulo/core/data/ConditionalMutation.html
+[getStatus]: {{ site.javadoc_core }}/org/apache/accumulo/core/client/ConditionalWriter.Result.html#getStatus()
+[write]: {{ site.javadoc_core }}/org/apache/accumulo/core/client/ConditionalWriter.html#write(org.apache.accumulo.core.data.ConditionalMutation)
+[setValue]: {{ site.javadoc_core }}/org/apache/accumulo/core/data/Condition.html#setValue(java.lang.CharSequence)
+[ACCEPTED]: {{ site.javadoc_core }}/org/apache/accumulo/core/client/ConditionalWriter.Status.html#ACCEPTED

-- 
To stop receiving notification emails like this one, please contact
['"commits@accumulo.apache.org" <commits@accumulo.apache.org>'].

Mime
View raw message