Return-Path: X-Original-To: archive-asf-public-internal@cust-asf2.ponee.io Delivered-To: archive-asf-public-internal@cust-asf2.ponee.io Received: from cust-asf.ponee.io (cust-asf.ponee.io [163.172.22.183]) by cust-asf2.ponee.io (Postfix) with ESMTP id DD50C200C1A for ; Mon, 13 Feb 2017 23:03:57 +0100 (CET) Received: by cust-asf.ponee.io (Postfix) id DBD5B160B60; Mon, 13 Feb 2017 22:03:57 +0000 (UTC) Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by cust-asf.ponee.io (Postfix) with SMTP id B3D3C160B4A for ; Mon, 13 Feb 2017 23:03:56 +0100 (CET) Received: (qmail 43231 invoked by uid 500); 13 Feb 2017 22:03:56 -0000 Mailing-List: contact commits-help@geode.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@geode.apache.org Delivered-To: mailing list commits@geode.apache.org Received: (qmail 43222 invoked by uid 99); 13 Feb 2017 22:03:55 -0000 Received: from git1-us-west.apache.org (HELO git1-us-west.apache.org) (140.211.11.23) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 13 Feb 2017 22:03:55 +0000 Received: by git1-us-west.apache.org (ASF Mail Server at git1-us-west.apache.org, from userid 33) id CE5AFDFC2F; Mon, 13 Feb 2017 22:03:55 +0000 (UTC) Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit From: kmiller@apache.org To: commits@geode.apache.org Message-Id: <04068f1b78bc41759407a02af78fa324@git.apache.org> X-Mailer: ASF-Git Admin Mailer Subject: geode-examples git commit: GEODE-2231 Revise Producer tests and the README.md Date: Mon, 13 Feb 2017 22:03:55 +0000 (UTC) archived-at: Mon, 13 Feb 2017 22:03:58 -0000 Repository: geode-examples Updated Branches: refs/heads/feature/GEODE-2231 b90991e1f -> c22885346 GEODE-2231 Revise Producer tests and the README.md - Re-run spotlessApply. - Clean up unused code and update comments. Project: http://git-wip-us.apache.org/repos/asf/geode-examples/repo Commit: http://git-wip-us.apache.org/repos/asf/geode-examples/commit/c2288534 Tree: http://git-wip-us.apache.org/repos/asf/geode-examples/tree/c2288534 Diff: http://git-wip-us.apache.org/repos/asf/geode-examples/diff/c2288534 Branch: refs/heads/feature/GEODE-2231 Commit: c22885346b8af173bc4765cf179b0a96d70959bc Parents: b90991e Author: Karen Miller Authored: Mon Feb 13 14:00:27 2017 -0800 Committer: Karen Miller Committed: Mon Feb 13 14:00:27 2017 -0800 ---------------------------------------------------------------------- partitioned/README.md | 184 +++++++++++-------- .../examples/partitioned/BadEmployeeKey.java | 4 - .../geode/examples/partitioned/Consumer.java | 8 +- .../examples/partitioned/EmployeeData.java | 2 - .../geode/examples/partitioned/EmployeeKey.java | 4 - .../geode/examples/partitioned/Producer.java | 20 +- .../examples/partitioned/ConsumerTest.java | 29 +-- .../examples/partitioned/ProducerTest.java | 36 ++-- 8 files changed, 146 insertions(+), 141 deletions(-) ---------------------------------------------------------------------- http://git-wip-us.apache.org/repos/asf/geode-examples/blob/c2288534/partitioned/README.md ---------------------------------------------------------------------- diff --git a/partitioned/README.md b/partitioned/README.md index 1c51058..748a919 100644 --- a/partitioned/README.md +++ b/partitioned/README.md @@ -17,7 +17,7 @@ limitations under the License. # Geode partitioned region example -This basic example demonstrates the basic properties of partitioned regions: +This example demonstrates the basic properties of partitioned regions: - Data entries are distributed across all servers that host a region. The distribution is like database sharding, except that the distribution @@ -50,113 +50,143 @@ This example is a simple demonstration of some basic Geode APIs, as well as providing ```gfsh``` command examples. ## Steps -1. From the ```geode-examples/partitioned``` directory, -build the jar (with the ```EmployeeKey```, ```BadEmployeeKey```, -and ```EmployeeData``` classes): - - $ ../gradlew build +1. Set directory ```geode-examples/partitioned``` to be the +current working directory. +Each step in this example specifies paths relative to that directory. -1. From the ```geode-examples/partitioned``` directory, -run a script that starts a locator and two servers. -The JAR built in the first step is placed -onto the classpath when starting the servers: +1. Build the jar (with the ```EmployeeKey```, ```BadEmployeeKey```, +and ```EmployeeData``` classes): - $ scripts/startAll.sh + ``` + $ ../gradlew build + ``` +1. Run a script that starts a locator and two servers. +The built JAR will be placed onto the classpath when the script +starts the servers: + ``` + $ scripts/startAll.sh + ``` Each of the servers hosts both partitioned regions. -1. Run the producer to put the same 10 entries into both the -```EmployeeRegion``` and the ```BadEmployeeRegion```: + +1. Run the producer to put the same 10 entries into both the ```EmployeeRegion``` and the ```BadEmployeeRegion```: + + ``` + $ ../gradlew run -Pmain=Producer + ... + ... + INFO: Inserted 10 entries in EmployeeRegion. + INFO: Inserted 10 entries in BadEmployeeRegion. + ``` - $ ../gradlew run -Pmain=Producer - ... - ... - INFO: Inserted 10 entries in EmployeeRegion. - INFO: Inserted 10 entries in BadEmployeeRegion. To see contents of the region keys, use a ```gfsh``` query: - - $ $GEODE_HOME/bin/gfsh - ... - gfsh>connect - gfsh>query --query="select e.key from /EmployeeRegion.entries e" - Or, to see contents of the region values, use a ```gfsh``` query: - gfsh>query --query="select * from /EmployeeRegion" -1. Run the consumer to get and log all 10 entries in each region, -the```EmployeeRegion``` and the ```BadEmployeeRegion```: + ``` + $ $GEODE_HOME/bin/gfsh + ... + gfsh>connect + gfsh>query --query="select e.key from /EmployeeRegion.entries e" + ``` - $ ../gradlew run -Pmain=Consumer + Or, to see contents of the region values, use a ```gfsh``` query: - Note that the quantity of entries may also be observed with ```gfsh```: - - $ $GEODE_HOME/bin/gfsh - ... - gfsh>connect - gfsh>describe region --name=EmployeeRegion - .......................................................... - Name : EmployeeRegion - Data Policy : partition - Hosting Members : server2 - server1 + ``` + gfsh>query --query="select * from /EmployeeRegion" + ``` - Non-Default Attributes Shared By Hosting Members +1. Run the consumer to get and print all 10 entries in each region, the +```EmployeeRegion``` and the ```BadEmployeeRegion```: - Type | Name | Value - ------ | ----------- | --------- - Region | size | 10 - | data-policy | PARTITION + ``` + $ ../gradlew run -Pmain=Consumer + ``` - gfsh>quit + Note that the quantity of entries may also be observed with ```gfsh```: + + ``` + $ $GEODE_HOME/bin/gfsh + ... + gfsh>connect + gfsh>describe region --name=EmployeeRegion + .......................................................... + Name : EmployeeRegion + Data Policy : partition + Hosting Members : server2 + server1 + + Non-Default Attributes Shared By Hosting Members + + Type | Name | Value + ------ | ----------- | --------- + Region | size | 10 + | data-policy | PARTITION + + gfsh>quit + ``` As an alternative, ```gfsh``` maybe used to identify how many entries there are for each region on each server by looking at statistics. - gfsh>show metrics --categories=partition --region=/BadEmployeeRegion --member=server1 - + ``` + gfsh>show metrics --categories=partition --region=/BadEmployeeRegion --member=server1 + ``` Within the output, the result for ```totalBucketSize``` identifies the number of entries hosted on the specified server. Vary the command to see both ```server1``` and ```server2```, as well as ```EmployeeRegion``` and ```BadEmployeeRegion```. - + Note that for the ```BadEmployeeRegion```, one of the servers will host all the entries, while the other server will not have any of the entries. This is due to the bad hash code generated for those keys. -1. Kill one of the servers: - - $ $GEODE_HOME/bin/gfsh - ... - gfsh>connect - gfsh>stop server --name=server1 - gfsh>quit -5. Run the consumer a second time, and notice that only approximately half of -the entries of the ```EmployeeRegion``` are still available: - - $ ../gradlew run -Pmain=Consumer - ... - ... - INFO: Done. 6 entries available on the server(s). +1. Kill one of the servers: + ``` + $ $GEODE_HOME/bin/gfsh + ... + gfsh>connect + gfsh>stop server --name=server1 + gfsh>quit + ``` + +1. Run the consumer a second time, and notice that approximately half of +the entries of the ```EmployeeRegion``` are still available on the remaining server. +Those hosted by the server that was stopped were lost. +And, depending on which server hosted all the `BadEmployeeRegion` entries and which server was stopped, there will either be 0 entries or all 10 entries on the remaining server. + + ``` + $ ../gradlew run -Pmain=Consumer + ... + ... + INFO: 6 entries in EmployeeRegion on the server(s). + ... + INFO: 0 entries in BadEmployeeRegion on the server(s). + ``` Again, this observation may also be made with ```gfsh```: - $ $GEODE_HOME/bin/gfsh - ... - gfsh>connect - gfsh>describe region --name=EmployeeRegion - .......................................................... - Name : EmployeeRegion - Data Policy : partition - Hosting Members : server2 + ``` + $ $GEODE_HOME/bin/gfsh + ... + gfsh>connect + gfsh>describe region --name=EmployeeRegion + .......................................................... + Name : EmployeeRegion + Data Policy : partition + Hosting Members : server2 - Non-Default Attributes Shared By Hosting Members + Non-Default Attributes Shared By Hosting Members - Type | Name | Value - ------ | ----------- | --------- - Region | size | 4 - | data-policy | PARTITION + Type | Name | Value + ------ | ----------- | --------- + Region | size | 4 + | data-policy | PARTITION - gfsh>quit + gfsh>quit + ``` 6. Shut down the system: - $ scripts/stopAll.sh + ``` + $ scripts/stopAll.sh + ``` http://git-wip-us.apache.org/repos/asf/geode-examples/blob/c2288534/partitioned/src/main/java/org/apache/geode/examples/partitioned/BadEmployeeKey.java ---------------------------------------------------------------------- diff --git a/partitioned/src/main/java/org/apache/geode/examples/partitioned/BadEmployeeKey.java b/partitioned/src/main/java/org/apache/geode/examples/partitioned/BadEmployeeKey.java index cf3f98a..31c2610 100644 --- a/partitioned/src/main/java/org/apache/geode/examples/partitioned/BadEmployeeKey.java +++ b/partitioned/src/main/java/org/apache/geode/examples/partitioned/BadEmployeeKey.java @@ -14,10 +14,6 @@ */ package org.apache.geode.examples.partitioned; -import java.util.Objects; -import java.util.logging.Logger; -import java.io.Serializable; -import org.apache.geode.cache.client.ClientCache; public class BadEmployeeKey extends EmployeeKey { http://git-wip-us.apache.org/repos/asf/geode-examples/blob/c2288534/partitioned/src/main/java/org/apache/geode/examples/partitioned/Consumer.java ---------------------------------------------------------------------- diff --git a/partitioned/src/main/java/org/apache/geode/examples/partitioned/Consumer.java b/partitioned/src/main/java/org/apache/geode/examples/partitioned/Consumer.java index 9fd47a5..1cef4e9 100644 --- a/partitioned/src/main/java/org/apache/geode/examples/partitioned/Consumer.java +++ b/partitioned/src/main/java/org/apache/geode/examples/partitioned/Consumer.java @@ -33,9 +33,7 @@ public class Consumer extends BaseClient { public void printRegionContents() { - /* - * Print EmployeeRegion size and contents - */ + /* Print EmployeeRegion size and contents */ Region r1 = this.getRegion1(); Set setOfKeys1 = r1.keySetOnServer(); logger.info(setOfKeys1.size() + " entries in EmployeeRegion on the server(s)."); @@ -46,9 +44,7 @@ public class Consumer extends BaseClient { } } - /* - * Print BadEmployeeRegion size and contents - */ + /* Print BadEmployeeRegion size and contents */ Region r2 = this.getRegion2(); Set setOfKeys2 = r2.keySetOnServer(); logger.info(setOfKeys2.size() + " entries in BadEmployeeRegion on the server(s)."); http://git-wip-us.apache.org/repos/asf/geode-examples/blob/c2288534/partitioned/src/main/java/org/apache/geode/examples/partitioned/EmployeeData.java ---------------------------------------------------------------------- diff --git a/partitioned/src/main/java/org/apache/geode/examples/partitioned/EmployeeData.java b/partitioned/src/main/java/org/apache/geode/examples/partitioned/EmployeeData.java index a53ec99..c79962d 100644 --- a/partitioned/src/main/java/org/apache/geode/examples/partitioned/EmployeeData.java +++ b/partitioned/src/main/java/org/apache/geode/examples/partitioned/EmployeeData.java @@ -14,14 +14,12 @@ */ package org.apache.geode.examples.partitioned; -import java.util.logging.Logger; import java.io.Serializable; public class EmployeeData implements Serializable { private static final long serialVersionUID = 1L; - static final Logger logger = Logger.getAnonymousLogger(); private EmployeeKey nameAndNumber; private int salary; private int hoursPerWeek; http://git-wip-us.apache.org/repos/asf/geode-examples/blob/c2288534/partitioned/src/main/java/org/apache/geode/examples/partitioned/EmployeeKey.java ---------------------------------------------------------------------- diff --git a/partitioned/src/main/java/org/apache/geode/examples/partitioned/EmployeeKey.java b/partitioned/src/main/java/org/apache/geode/examples/partitioned/EmployeeKey.java index 1899e9e..7581676 100644 --- a/partitioned/src/main/java/org/apache/geode/examples/partitioned/EmployeeKey.java +++ b/partitioned/src/main/java/org/apache/geode/examples/partitioned/EmployeeKey.java @@ -14,16 +14,12 @@ */ package org.apache.geode.examples.partitioned; -import java.util.Objects; -import java.util.logging.Logger; import java.io.Serializable; -import org.apache.geode.cache.client.ClientCache; public class EmployeeKey implements Serializable { private static final long serialVersionUID = 1L; - static final Logger logger = Logger.getAnonymousLogger(); private String name; private int emplNumber; http://git-wip-us.apache.org/repos/asf/geode-examples/blob/c2288534/partitioned/src/main/java/org/apache/geode/examples/partitioned/Producer.java ---------------------------------------------------------------------- diff --git a/partitioned/src/main/java/org/apache/geode/examples/partitioned/Producer.java b/partitioned/src/main/java/org/apache/geode/examples/partitioned/Producer.java index 4869e01..36f2044 100644 --- a/partitioned/src/main/java/org/apache/geode/examples/partitioned/Producer.java +++ b/partitioned/src/main/java/org/apache/geode/examples/partitioned/Producer.java @@ -16,7 +16,6 @@ package org.apache.geode.examples.partitioned; import org.apache.geode.cache.client.ClientCache; import org.apache.geode.cache.Region; -import org.apache.geode.cache.client.ServerOperationException; public class Producer extends BaseClient { @@ -33,17 +32,13 @@ public class Producer extends BaseClient { this.clientCache = clientCache; } + /* Put 10 entries into the region with the quality hashing function. */ public void populateRegion() { Region r = getRegion1(); EmployeeKey k1 = new EmployeeKey("Alex Able", 160); EmployeeData d1 = new EmployeeData(k1, 70000, 40); - try { - r.put(k1, d1); - } catch (ServerOperationException e) { - logger.info("ServerOperationException " + e.getMessage()); - logger.info("ServerOperationException " + e.getCause()); - } + r.put(k1, d1); EmployeeKey k2 = new EmployeeKey("Bertie Bell", 170); EmployeeData d2 = new EmployeeData(k2, 72000, 40); @@ -84,17 +79,16 @@ public class Producer extends BaseClient { logger.info("Inserted 10 entries in EmployeeRegion."); } + /* + * Put 10 entries into the region with the bad hashing function. The entries are the same as those + * put into the region with the quality hashing function. + */ public void populateBadRegion() { Region r = getRegion2(); BadEmployeeKey k1 = new BadEmployeeKey("Alex Able", 160); EmployeeData d1 = new EmployeeData(k1, 70000, 40); - try { - r.put(k1, d1); - } catch (ServerOperationException e) { - logger.info("ServerOperationException " + e.getMessage()); - logger.info("ServerOperationException " + e.getCause()); - } + r.put(k1, d1); BadEmployeeKey k2 = new BadEmployeeKey("Bertie Bell", 170); EmployeeData d2 = new EmployeeData(k2, 72000, 40); http://git-wip-us.apache.org/repos/asf/geode-examples/blob/c2288534/partitioned/src/test/java/org/apache/geode/examples/partitioned/ConsumerTest.java ---------------------------------------------------------------------- diff --git a/partitioned/src/test/java/org/apache/geode/examples/partitioned/ConsumerTest.java b/partitioned/src/test/java/org/apache/geode/examples/partitioned/ConsumerTest.java index e0b3709..1682ab8 100644 --- a/partitioned/src/test/java/org/apache/geode/examples/partitioned/ConsumerTest.java +++ b/partitioned/src/test/java/org/apache/geode/examples/partitioned/ConsumerTest.java @@ -31,9 +31,6 @@ import org.junit.rules.ExpectedException; import org.apache.geode.cache.Region; import org.apache.geode.cache.client.ClientCache; -import org.apache.geode.cache.client.NoAvailableLocatorsException; -import org.mockito.invocation.InvocationOnMock; -import org.mockito.stubbing.Answer; public class ConsumerTest { @@ -52,8 +49,6 @@ public class ConsumerTest { when(region1.getName()).thenReturn(Consumer.REGION1_NAME); when(region2.getName()).thenReturn(Consumer.REGION2_NAME); when(keys.size()).thenReturn(Consumer.NUM_ENTRIES); - // when(region1.keySetOnServer()).thenReturn(keys); - // when(region2.keySetOnServer()).thenReturn(keys); when(clientCache.createClientRegionFactory(ClientRegionShortcut.PROXY)) .thenReturn(clientRegionFactory); when(clientRegionFactory.create(Consumer.REGION1_NAME)).thenReturn(region1); @@ -67,11 +62,11 @@ public class ConsumerTest { EmployeeKey k2 = new EmployeeKey("Toni Tiptoe", 180); EmployeeData d2 = new EmployeeData(k2, 70000, 40); emplMap.put(k2, d2); - /* Mock the region keySetOnServer,size, and get methods with values from the map */ + /* Use HashMap as fake region for keySetOnServer, size, and get methods */ when(region1.keySetOnServer()).thenReturn(emplMap.keySet()); when(region1.size()).thenReturn(emplMap.size()); - when(region1.get(eq(k1))).thenReturn(emplMap.get(d1)); - when(region1.get(eq(k2))).thenReturn(emplMap.get(d2)); + when(region1.get(eq(k1))).thenReturn(emplMap.get(k1)); + when(region1.get(eq(k2))).thenReturn(emplMap.get(k2)); Map badEmplMap = new HashMap<>(); BadEmployeeKey bk1 = new BadEmployeeKey("Bertie Bell", 170); @@ -80,20 +75,11 @@ public class ConsumerTest { BadEmployeeKey bk2 = new BadEmployeeKey("Toni Tiptoe", 180); EmployeeData bd2 = new EmployeeData(bk2, 70000, 40); badEmplMap.put(bk2, bd2); - /* Mock the region keySetOnServer,size, and get methods with values from the map */ + /* Use HashMap as fake region for keySetOnServer, size, and get methods */ when(region2.keySetOnServer()).thenReturn(badEmplMap.keySet()); when(region2.size()).thenReturn(badEmplMap.size()); - when(region2.get(eq(bk1))).thenReturn(bd1); - when(region2.get(eq(bk2))).thenReturn(bd2); - - // when(region1.get(any())).then(new Answer() { - // - // @Override - // public Object answer(InvocationOnMock invocation) throws Throwable { - // EmployeeKey key = invocation.getArgumentAt(0, EmployeeKey.class); - // return emplMap.get(key); - // } - // }); + when(region2.get(eq(bk1))).thenReturn(badEmplMap.get(bk1)); + when(region2.get(eq(bk2))).thenReturn(badEmplMap.get(bk2)); consumer = new Consumer(clientCache); } @@ -109,9 +95,6 @@ public class ConsumerTest { consumer.getRegion1().getName()); } - /* - * Doesn't work because mocked clientRegionFactory.create(any()) returns region1, not region2 - */ @Test public void testConsumerGetRegion2() { assertEquals("Region names do not match", Consumer.REGION2_NAME, http://git-wip-us.apache.org/repos/asf/geode-examples/blob/c2288534/partitioned/src/test/java/org/apache/geode/examples/partitioned/ProducerTest.java ---------------------------------------------------------------------- diff --git a/partitioned/src/test/java/org/apache/geode/examples/partitioned/ProducerTest.java b/partitioned/src/test/java/org/apache/geode/examples/partitioned/ProducerTest.java index d794820..70729bc 100644 --- a/partitioned/src/test/java/org/apache/geode/examples/partitioned/ProducerTest.java +++ b/partitioned/src/test/java/org/apache/geode/examples/partitioned/ProducerTest.java @@ -19,6 +19,9 @@ import static org.mockito.Mockito.*; import java.util.Set; +import org.apache.geode.cache.client.ClientRegionFactory; +import org.apache.geode.cache.client.ClientRegionShortcut; +import static org.junit.Assert.*; import org.junit.After; import org.junit.Before; import org.junit.Rule; @@ -35,30 +38,39 @@ public class ProducerTest { private Producer producer; private ClientCache clientCache = mock(ClientCache.class); - private Region region = mock(Region.class); + private Region region1 = mock(Region.class); + private Region region2 = mock(Region.class); + private ClientRegionFactory clientRegionFactory = mock(ClientRegionFactory.class); private Set keys = mock(Set.class); + @Before - public void setup() throws Exception { - when(region.getName()).thenReturn(Producer.REGION1_NAME); - when(region.keySetOnServer()).thenReturn(keys); - when(clientCache.getRegion(any())).thenReturn(region); + public void setup() { + when(region1.getName()).thenReturn(Producer.REGION1_NAME); + when(region2.getName()).thenReturn(Producer.REGION2_NAME); + when(keys.size()).thenReturn(Producer.NUM_ENTRIES); + when(clientCache.createClientRegionFactory(ClientRegionShortcut.PROXY)) + .thenReturn(clientRegionFactory); + when(clientRegionFactory.create(Consumer.REGION1_NAME)).thenReturn(region1); + when(clientRegionFactory.create(Consumer.REGION2_NAME)).thenReturn(region2); + } + @Test - public void populateRegionShouldReturnCorrectNumberOfEntries() throws Exception { + public void testPopulateRegion() { producer = new Producer(clientCache); - producer.setRegion1(region); - producer.populateRegion(); - verify(region, times(producer.NUM_ENTRIES)).put(any(), any()); + verify(region1, times(10)).put(any(), any()); + } @Test - public void populateWhenRegionDoesNotExistShouldThrowNullPointer() throws Exception { + public void testPopulateBadRegion() { producer = new Producer(clientCache); - expectedException.expect(NullPointerException.class); - producer.populateRegion(); + producer.populateBadRegion(); + verify(region2, times(10)).put(any(), any()); + } @After