geode-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From kmil...@apache.org
Subject geode-examples git commit: GEODE-2231 Rework README, code, and tests to present partitioning example in two parts
Date Thu, 23 Feb 2017 00:22:11 GMT
Repository: geode-examples
Updated Branches:
  refs/heads/feature/GEODE-2231 17fd0ab82 -> 912a37234


GEODE-2231 Rework README, code, and tests to present
partitioning example in two parts


Project: http://git-wip-us.apache.org/repos/asf/geode-examples/repo
Commit: http://git-wip-us.apache.org/repos/asf/geode-examples/commit/912a3723
Tree: http://git-wip-us.apache.org/repos/asf/geode-examples/tree/912a3723
Diff: http://git-wip-us.apache.org/repos/asf/geode-examples/diff/912a3723

Branch: refs/heads/feature/GEODE-2231
Commit: 912a37234fdccb9a80de9335bab366b5cd3ec6b0
Parents: 17fd0ab
Author: Karen Miller <kmiller@pivotal.io>
Authored: Wed Feb 22 16:21:06 2017 -0800
Committer: Karen Miller <kmiller@pivotal.io>
Committed: Wed Feb 22 16:21:06 2017 -0800

----------------------------------------------------------------------
 partitioned/README.md                           | 153 +++++++++++++------
 .../geode/examples/partitioned/BaseClient.java  |   4 +
 .../geode/examples/partitioned/Consumer.java    |  24 ++-
 .../geode/examples/partitioned/Producer.java    |  25 ++-
 .../examples/partitioned/ConsumerTest.java      |  55 +++++++
 .../examples/partitioned/ProducerTest.java      |  52 +++++++
 6 files changed, 261 insertions(+), 52 deletions(-)
----------------------------------------------------------------------


http://git-wip-us.apache.org/repos/asf/geode-examples/blob/912a3723/partitioned/README.md
----------------------------------------------------------------------
diff --git a/partitioned/README.md b/partitioned/README.md
index 7b2e4b0..0a90284 100644
--- a/partitioned/README.md
+++ b/partitioned/README.md
@@ -79,6 +79,17 @@ The argument specification identifies the region.
     INFO: Inserted 10 entries in EmployeeRegion.
     ```
 
+1. There are several ways to see the contents of the region.
+Run the consumer to get and print all 10 entries in the
+```EmployeeRegion```.  The argument specification identifies the region.
+
+    ```
+    $ ../gradlew run -Pmain=Consumer -Pargs=EmployeeRegion
+    ...
+    INFO: 10 entries in EmployeeRegion on the server(s).
+    ...
+    ```
+
     To see contents of the region keys, use a ```gfsh``` query:
 
     ```
@@ -86,27 +97,20 @@ The argument specification identifies the region.
     ...
     gfsh>connect
     gfsh>query --query="select e.key from /EmployeeRegion.entries e"
+    ...
     ```
 
+
     Or, to see contents of the region values, use a ```gfsh``` query:
 
     ```
     gfsh>query --query="select * from /EmployeeRegion"
-    ```
-
-1. Run the consumer to get and print all 10 entries in each region, the
-```EmployeeRegion``` and the ```BadEmployeeRegion```:
-
-    ```
-    $ ../gradlew run -Pmain=Consumer
+    ...
     ```
 
     Note that the quantity of entries may also be observed with ```gfsh```:
  
     ```
-    $ $GEODE_HOME/bin/gfsh
-    ...
-    gfsh>connect
     gfsh>describe region --name=EmployeeRegion
     ..........................................................
     Name            : EmployeeRegion
@@ -128,18 +132,16 @@ The argument specification identifies the region.
     there are for each region on each server by looking at statistics.
 
     ```
-    gfsh>show metrics --categories=partition --region=/BadEmployeeRegion --member=server1
+    gfsh>show metrics --categories=partition --region=/EmployeeRegion --member=server1
     ```
     Within the output, the result for ```totalBucketSize``` identifies
     the number of entries hosted on the specified server.
-    Vary the command to see both ```server1``` and ```server2```, as well as
-    ```EmployeeRegion``` and ```BadEmployeeRegion```.
+    Vary the command to see statistics for both ```server1``` and
+    ```server2```.
     
-    Note that for the ```BadEmployeeRegion```, one of the servers will host
-    all the entries, while the other server will not have any of the entries.
-    This is due to the bad hash code generated for those keys.
 
-1. Kill one of the servers:
+1. The region entries are distributed across both servers.
+Kill one of the servers:
 
     ```
     $ $GEODE_HOME/bin/gfsh
@@ -150,38 +152,102 @@ The argument specification identifies the region.
     ```
 
 1. Run the consumer a second time, and notice that approximately half of
-the entries of the ```EmployeeRegion``` are still available on the remaining server.
+the entries of the ```EmployeeRegion``` are still available on the 
+remaining server.
 Those hosted by the server that was stopped were lost.
-And, depending on which server hosted all the `BadEmployeeRegion` entries and which server
was stopped, there will either be 0 entries or all 10 entries on the remaining server.
 
     ```
-    $ ../gradlew run -Pmain=Consumer
+    $ ../gradlew run -Pmain=Consumer -Pargs=EmployeeRegion
     ...
     ...
-    INFO: 6 entries in EmployeeRegion on the server(s).
+    INFO: 5 entries in EmployeeRegion on the server(s).
     ...
-    INFO: 0 entries in BadEmployeeRegion on the server(s).
     ```
-    Again, this observation may also be made with ```gfsh```:
+
+6. Shut down the system:
+
+    ```
+    $ scripts/stopAll.sh
+    ```
+
+## Part 2: What Can Go Wrong
+
+Hashing distributes entries among buckets that reside on servers.
+A good hash code is important in order to spread the entries among buckets.
+
+The ```EmployeeRegion``` used in Part 1 of this example has a good hashing
+function.
+Region entries are well distributed among buckets
+(and therefore, among servers).
+The ```BadEmployeeRegion``` used in this part of the example
+has a pointedly poor hash code implementation,
+to illustrate what can go wrong.
+The hash code is so bad that all entries in the
+```BadEmployeeRegion``` end up in the same bucket.
+Because of this,
+one of the servers will host all the entries,
+while the other server will not have any of the entries.
+
+Here are the steps to run through this example again, 
+using ```BadEmployeeRegion``` instead of ```EmployeeRegion```.
+This part assumes that you have already built the JAR (step 2 in Part 1 
+of this example).
+
+1. Set directory ```geode-examples/partitioned``` to be the
+current working directory.
+Each step in this example specifies paths relative to that directory.
+
+1. Run a script that starts a locator and two servers.
+The built JAR will be placed onto the classpath when the script 
+starts the servers:
+
+    ```
+    $ scripts/startAll.sh
+    ```
+
+1. Run the producer to put 10 entries into the ```BadEmployeeRegion```.
+The argument specification identifies the region.
+
+    ```
+    $ ../gradlew run -Pmain=Producer -Pargs=BadEmployeeRegion
+    ... 
+    INFO: Inserted 10 entries in BadEmployeeRegion.
+    ```
+
+1.  Run the consumer to get and print all 10 entries in the
+`BadEmployeeRegion`.  The argument specification identifies the region.
+Alternatively, use one of the `gfsh` commands 
+(given in Part 1 of this example)
+to verify that the servers are hosting the 10 entries in the region.
+
+    ```
+    $ ../gradlew run -Pmain=Consumer -Pargs=EmployeeRegion
+    ...
+    INFO: 10 entries in EmployeeRegion on the server(s).
+    ...
+    ```
+
+1.  Kill one of the servers:
 
     ```
     $ $GEODE_HOME/bin/gfsh
     ...
     gfsh>connect
-    gfsh>describe region --name=EmployeeRegion
-    ..........................................................
-    Name            : EmployeeRegion
-    Data Policy     : partition
-    Hosting Members : server2
-
-    Non-Default Attributes Shared By Hosting Members  
+    gfsh>stop server --name=server1
+    gfsh>quit
+    ```
 
-     Type  |    Name     | Value
-    ------ | ----------- | ---------
-    Region | size        | 4
-           | data-policy | PARTITION
+1. Run the consumer or a `gfsh` query, and notice that either all or
+none of the entries of the ```BadEmployeeRegion``` are still available on the 
+remaining server.
+Those hosted by the server that was stopped were lost.
 
-    gfsh>quit
+    ```
+    $ ../gradlew run -Pmain=Consumer -Pargs=BadEmployeeRegion
+    ...
+    ...
+    INFO: 5 entries in BadEmployeeRegion on the server(s).
+    ...
     ```
 
 6. Shut down the system:
@@ -190,13 +256,14 @@ And, depending on which server hosted all the `BadEmployeeRegion` entries
and wh
     $ scripts/stopAll.sh
     ```
 
-## Part 2: What Can Go Wrong
+## Partitioning Example Takeaways
 
-Hashing distributes entries among buckets that reside on servers.
-A good hash code is important in order to spread the entries among buckets.
+1. Partitioned regions distribute region entries across buckets within servers.
+A robust system will use redundancy in conjunction with partitioning
+in production systems,
+so that data is not lost if a server goes down.
+
+2. A proper hashcode is important for distributing entries across buckets.
+Not demonstrated in this example, but a proper equals methods is also
+required.
 
-While the ```EmployeeRegion``` used in Part 1 of this example
-has a good hashing function,
-the ```BadEmployeeRegion``` has a pointedly poor hash code implementation.
-The hash code is so bad that all entries in the
-```BadEmployeeRegion``` end up in the same bucket.

http://git-wip-us.apache.org/repos/asf/geode-examples/blob/912a3723/partitioned/src/main/java/org/apache/geode/examples/partitioned/BaseClient.java
----------------------------------------------------------------------
diff --git a/partitioned/src/main/java/org/apache/geode/examples/partitioned/BaseClient.java
b/partitioned/src/main/java/org/apache/geode/examples/partitioned/BaseClient.java
index 5e51a66..5fdc236 100644
--- a/partitioned/src/main/java/org/apache/geode/examples/partitioned/BaseClient.java
+++ b/partitioned/src/main/java/org/apache/geode/examples/partitioned/BaseClient.java
@@ -47,6 +47,10 @@ public abstract class BaseClient {
     this.clientCache = getClientCache();
   }
 
+  public BaseClient(ClientCache clientCache) {
+    this.clientCache = clientCache;
+  }
+
   protected Region getRegion1() {
     if (region1 == null) {
       region1 = getClientCache()

http://git-wip-us.apache.org/repos/asf/geode-examples/blob/912a3723/partitioned/src/main/java/org/apache/geode/examples/partitioned/Consumer.java
----------------------------------------------------------------------
diff --git a/partitioned/src/main/java/org/apache/geode/examples/partitioned/Consumer.java
b/partitioned/src/main/java/org/apache/geode/examples/partitioned/Consumer.java
index 1cef4e9..2e61866 100644
--- a/partitioned/src/main/java/org/apache/geode/examples/partitioned/Consumer.java
+++ b/partitioned/src/main/java/org/apache/geode/examples/partitioned/Consumer.java
@@ -22,7 +22,27 @@ public class Consumer extends BaseClient {
 
   public static void main(String[] args) {
     Consumer c = new Consumer();
-    c.printRegionContents();
+    c.checkAndPrint(args);
+  }
+
+  public void checkAndPrint(String[] args) {
+    if (0 == args.length) {
+      throw new RuntimeException("Expected argument specifying region name.");
+    } else {
+      if (args.length > 1) {
+        throw new RuntimeException("Expected only 1 argument, and received more than 1.");
+      } else {
+        if (args[0].equals("EmployeeRegion")) {
+          this.printRegionContents();
+        } else {
+          if (args[0].equals("BadEmployeeRegion")) {
+            this.printBadRegionContents();
+          } else {
+            throw new RuntimeException("Unrecognized region name in argument specification.");
+          }
+        }
+      }
+    }
   }
 
   public Consumer() {}
@@ -43,7 +63,9 @@ public class Consumer extends BaseClient {
         logger.info(r1.get(key).toString());
       }
     }
+  }
 
+  public void printBadRegionContents() {
     /* Print BadEmployeeRegion size and contents */
     Region r2 = this.getRegion2();
     Set<EmployeeKey> setOfKeys2 = r2.keySetOnServer();

http://git-wip-us.apache.org/repos/asf/geode-examples/blob/912a3723/partitioned/src/main/java/org/apache/geode/examples/partitioned/Producer.java
----------------------------------------------------------------------
diff --git a/partitioned/src/main/java/org/apache/geode/examples/partitioned/Producer.java
b/partitioned/src/main/java/org/apache/geode/examples/partitioned/Producer.java
index 787625a..04fcfc1 100644
--- a/partitioned/src/main/java/org/apache/geode/examples/partitioned/Producer.java
+++ b/partitioned/src/main/java/org/apache/geode/examples/partitioned/Producer.java
@@ -20,27 +20,36 @@ import org.apache.geode.cache.Region;
 public class Producer extends BaseClient {
 
   public static void main(String[] args) {
-
     Producer p = new Producer();
+    p.checkAndPopulate(args);
+  }
+
+  public void checkAndPopulate(String[] args) {
     if (0 == args.length) {
       throw new RuntimeException("Expected argument specifying region name.");
     } else {
-      if (args[0].equals("EmployeeRegion")) {
-        p.populateRegion();
+      if (args.length > 1) {
+        throw new RuntimeException("Expected only 1 argument, and received more than 1.");
       } else {
-        if (args[0].equals("BadEmployeeRegion")) {
-          p.populateBadRegion();
+        if (args[0].equals("EmployeeRegion")) {
+          this.populateRegion();
         } else {
-          throw new RuntimeException("Unrecognized region name in argument specification.");
+          if (args[0].equals("BadEmployeeRegion")) {
+            this.populateBadRegion();
+          } else {
+            throw new RuntimeException("Unrecognized region name in argument specification.");
+          }
         }
       }
     }
   }
 
-  public Producer() {}
+  public Producer() {
+    super();
+  }
 
   public Producer(ClientCache clientCache) {
-    this.clientCache = clientCache;
+    super(clientCache);
   }
 
   /* Put 10 entries into the region with the quality hashing function. */

http://git-wip-us.apache.org/repos/asf/geode-examples/blob/912a3723/partitioned/src/test/java/org/apache/geode/examples/partitioned/ConsumerTest.java
----------------------------------------------------------------------
diff --git a/partitioned/src/test/java/org/apache/geode/examples/partitioned/ConsumerTest.java
b/partitioned/src/test/java/org/apache/geode/examples/partitioned/ConsumerTest.java
index 1682ab8..e6a550e 100644
--- a/partitioned/src/test/java/org/apache/geode/examples/partitioned/ConsumerTest.java
+++ b/partitioned/src/test/java/org/apache/geode/examples/partitioned/ConsumerTest.java
@@ -43,6 +43,12 @@ public class ConsumerTest {
   private Region region2 = mock(Region.class);
   private ClientRegionFactory clientRegionFactory = mock(ClientRegionFactory.class);
   private Set keys = mock(Set.class);
+  private static final String[] EMPTYARGS = new String[0];
+  private static final String[] GOODARGS1 = {"EmployeeRegion"};
+  private static final String[] GOODARGS2 = {"BadEmployeeRegion"};
+  private static final String[] BADARGS1 = {""};
+  private static final String[] BADARGS2 = {"BadRegionName"};
+  private static final String[] BADARGS3 = {"BadEmployeeRegion", "2"};
 
   @Before
   public void setup() {
@@ -106,4 +112,53 @@ public class ConsumerTest {
     consumer.printRegionContents();
   }
 
+  @Test
+  public void testPrintBadRegionContents() {
+    consumer.printBadRegionContents();
+  }
+
+  @Test
+  public void testCheckAndPrint1() {
+    /* no exception expected */
+    consumer.checkAndPrint(GOODARGS1);
+  }
+
+  @Test
+  public void testCheckAndPrint2() {
+    /* no exception expected */
+    consumer.checkAndPrint(GOODARGS2);
+  }
+
+  @Test
+  public void testCheckAndPrint3() {
+    /* no arguments specified, array length should be 0 */
+    expectedException.expect(Exception.class);
+    expectedException.expectMessage("Expected argument specifying region name.");
+    spy(Consumer.class).checkAndPrint(EMPTYARGS);
+  }
+
+  @Test
+  public void testCheckAndPrint4() {
+    /* First argument is the empty string */
+    expectedException.expect(Exception.class);
+    expectedException.expectMessage("Unrecognized region name in argument specification.");
+    spy(Consumer.class).checkAndPrint(BADARGS1);
+  }
+
+  @Test
+  public void testCheckAndPrint5() {
+    /* Arguments are an invalid region name */
+    expectedException.expect(Exception.class);
+    expectedException.expectMessage("Unrecognized region name in argument specification.");
+    spy(Consumer.class).checkAndPrint(BADARGS2);
+  }
+
+  @Test
+  public void testCheckAndPrint6() {
+    /* Arguments are an invalid region name and an extra but unused argument */
+    expectedException.expect(Exception.class);
+    expectedException.expectMessage("Expected only 1 argument, and received more than 1.");
+    spy(Consumer.class).checkAndPrint(BADARGS3);
+  }
+
 }

http://git-wip-us.apache.org/repos/asf/geode-examples/blob/912a3723/partitioned/src/test/java/org/apache/geode/examples/partitioned/ProducerTest.java
----------------------------------------------------------------------
diff --git a/partitioned/src/test/java/org/apache/geode/examples/partitioned/ProducerTest.java
b/partitioned/src/test/java/org/apache/geode/examples/partitioned/ProducerTest.java
index 70729bc..350d96d 100644
--- a/partitioned/src/test/java/org/apache/geode/examples/partitioned/ProducerTest.java
+++ b/partitioned/src/test/java/org/apache/geode/examples/partitioned/ProducerTest.java
@@ -37,11 +37,18 @@ public class ProducerTest {
   public ExpectedException expectedException = ExpectedException.none();
 
   private Producer producer;
+  private Producer pMock = mock(Producer.class);
   private ClientCache clientCache = mock(ClientCache.class);
   private Region<EmployeeKey, EmployeeData> region1 = mock(Region.class);
   private Region<BadEmployeeKey, EmployeeData> region2 = mock(Region.class);
   private ClientRegionFactory clientRegionFactory = mock(ClientRegionFactory.class);
   private Set keys = mock(Set.class);
+  private static final String[] EMPTYARGS = new String[0];
+  private static final String[] GOODARGS1 = {"EmployeeRegion"};
+  private static final String[] GOODARGS2 = {"BadEmployeeRegion"};
+  private static final String[] BADARGS1 = {""};
+  private static final String[] BADARGS2 = {"BadRegionName"};
+  private static final String[] BADARGS3 = {"BadEmployeeRegion", "2"};
 
 
   @Before
@@ -53,6 +60,7 @@ public class ProducerTest {
         .thenReturn(clientRegionFactory);
     when(clientRegionFactory.create(Consumer.REGION1_NAME)).thenReturn(region1);
     when(clientRegionFactory.create(Consumer.REGION2_NAME)).thenReturn(region2);
+    doNothing().when(pMock).populateRegion();
 
   }
 
@@ -73,6 +81,50 @@ public class ProducerTest {
 
   }
 
+  @Test
+  public void testCheckAndPopulate1() {
+    /* no exception expected */
+    pMock.checkAndPopulate(GOODARGS1);
+  }
+
+  @Test
+  public void testCheckAndPopulate2() {
+    /* no exception expected */
+    pMock.checkAndPopulate(GOODARGS2);
+  }
+
+  @Test
+  public void testCheckAndPopulate3() {
+    /* no arguments specified, array length should be 0 */
+    expectedException.expect(Exception.class);
+    expectedException.expectMessage("Expected argument specifying region name.");
+    spy(Producer.class).checkAndPopulate(EMPTYARGS);
+  }
+
+  @Test
+  public void testCheckAndPopulate4() {
+    /* First argument is the empty string */
+    expectedException.expect(Exception.class);
+    expectedException.expectMessage("Unrecognized region name in argument specification.");
+    spy(Producer.class).checkAndPopulate(BADARGS1);
+  }
+
+  @Test
+  public void testCheckAndPopulate5() {
+    /* Arguments are an invalid region name */
+    expectedException.expect(Exception.class);
+    expectedException.expectMessage("Unrecognized region name in argument specification.");
+    spy(Producer.class).checkAndPopulate(BADARGS2);
+  }
+
+  @Test
+  public void testCheckAndPopulate6() {
+    /* Arguments are an invalid region name and an extra but unused argument */
+    expectedException.expect(Exception.class);
+    expectedException.expectMessage("Expected only 1 argument, and received more than 1.");
+    spy(Producer.class).checkAndPopulate(BADARGS3);
+  }
+
   @After
   public void tearDown() {
 


Mime
View raw message