Mailing-List: contact issues-help@geode.incubator.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@geode.incubator.apache.org
Date: Tue, 27 Sep 2016 21:37:20 +0000 (UTC)
From: "Kenneth Howe (JIRA)" <jira@apache.org>
To: issues@geode.incubator.apache.org
Message-ID: <JIRA.12911942.1447181932000.678697.1475012240613@Atlassian.JIRA>
In-Reply-To: <JIRA.12911942.1447181932000@Atlassian.JIRA>
References: <JIRA.12911942.1447181932000@Atlassian.JIRA> <JIRA.12911942.1447181932419@arcas>
Subject: [jira] [Commented] (GEODE-538)
 PersistentColocatedPartitionedRegionDUnitTest.testRebalanceWithOfflineChildRegion
 failed if do not wait on getResult
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit
archived-at: Tue, 27 Sep 2016 21:37:26 -0000


    [ https://issues.apache.org/jira/browse/GEODE-538?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15527484#comment-15527484 ] 

Kenneth Howe commented on GEODE-538:
------------------------------------

With the async*.getResult(MAX_WAIT) commented out, then checkData() is called while persistent data recovery is still in progress following the child region creation on all the members. The test now (usually) fails because a PartitionedRegion.get() call targets a bucket that hasn't yet been recovered. PR.getNodeForBucketReadOrLoad() ends up creating a new bucket, and the get() ends up returning null.

Examining full logs there will also be one or more members that throw ConflictingPersistentDataException when the recovery thread(s) tries to recover the bucket that was erroneously created.

put() operations on PRs with incomplete persistent recovery can also cause this failure.

> PersistentColocatedPartitionedRegionDUnitTest.testRebalanceWithOfflineChildRegion failed if do not wait on getResult
> --------------------------------------------------------------------------------------------------------------------
>
>                 Key: GEODE-538
>                 URL: https://issues.apache.org/jira/browse/GEODE-538
>             Project: Geode
>          Issue Type: Bug
>          Components: persistence
>            Reporter: Jinmei Liao
>            Assignee: Kenneth Howe
>         Attachments: logs.zip
>
>
> While attempting to work on GEODE-506, I commented out line s1538-1540 like the following
>     async1 = vm1.invokeAsync(createChildPR);
>     async0 = vm0.invokeAsync(createChildPR);
>     AsyncInvocation async2 = vm2.invokeAsync(createChildPR);
>     //async0.getResult(MAX_WAIT);  -- line 1538
>     //async1.getResult(MAX_WAIT);  -- line 1539
>     //async2.getResult(MAX_WAIT);  -- line 1540
> This will reproduce the error similar to the one reported in GEODE506, but it failed on checking the data on parent region instead of the child region. Log files attached.
> dunit.RMIException: While invoking com.gemstone.gemfire.internal.cache.partitioned.PersistentPartitionedRegionTestBase$5.run in VM 0 running on Host localhost with 4 VMs
> 	at dunit.VM.invoke(VM.java:368)
> 	at dunit.VM.invoke(VM.java:312)
> 	at dunit.VM.invoke(VM.java:266)
> 	at com.gemstone.gemfire.internal.cache.partitioned.PersistentPartitionedRegionTestBase.checkData(PersistentPartitionedRegionTestBase.java:182)
> 	at com.gemstone.gemfire.internal.cache.partitioned.PersistentPartitionedRegionTestBase.checkData(PersistentPartitionedRegionTestBase.java:165)
> 	at com.gemstone.gemfire.internal.cache.partitioned.PersistentColocatedPartitionedRegionDUnitTest.rebalanceWithOfflineChildRegion(PersistentColocatedPartitionedRegionDUnitTest.java:1543)
> 	at com.gemstone.gemfire.internal.cache.partitioned.PersistentColocatedPartitionedRegionDUnitTest.testRebalanceWithOfflineChildRegion(PersistentColocatedPartitionedRegionDUnitTest.java:1223)
> 	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> 	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> 	at java.lang.reflect.Method.invoke(Method.java:497)
> 	at junit.framework.TestCase.runTest(TestCase.java:176)
> 	at junit.framework.TestCase.runBare(TestCase.java:141)
> 	at junit.framework.TestResult$1.protect(TestResult.java:122)
> 	at junit.framework.TestResult.runProtected(TestResult.java:142)
> 	at junit.framework.TestResult.run(TestResult.java:125)
> 	at junit.framework.TestCase.run(TestCase.java:129)
> 	at junit.framework.TestSuite.runTest(TestSuite.java:252)
> 	at junit.framework.TestSuite.run(TestSuite.java:247)
> 	at org.junit.internal.runners.JUnit38ClassRunner.run(JUnit38ClassRunner.java:86)
> 	at org.eclipse.jdt.internal.junit4.runner.JUnit4TestReference.run(JUnit4TestReference.java:86)
> 	at org.eclipse.jdt.internal.junit.runner.TestExecution.run(TestExecution.java:38)
> 	at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:459)
> 	at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:675)
> 	at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.run(RemoteTestRunner.java:382)
> 	at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.main(RemoteTestRunner.java:192)
> Caused by: junit.framework.AssertionFailedError: For key 1 expected:<a> but was:<null>
> 	at junit.framework.Assert.fail(Assert.java:57)
> 	at junit.framework.Assert.failNotEquals(Assert.java:329)
> 	at junit.framework.Assert.assertEquals(Assert.java:78)
> 	at junit.framework.TestCase.assertEquals(TestCase.java:244)
> 	at com.gemstone.gemfire.internal.cache.partitioned.PersistentPartitionedRegionTestBase$5.run(PersistentPartitionedRegionTestBase.java:177)
> 	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> 	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> 	at java.lang.reflect.Method.invoke(Method.java:497)
> 	at hydra.MethExecutor.executeObject(MethExecutor.java:267)
> 	at dunit.standalone.RemoteDUnitVM.executeMethodOnObject(RemoteDUnitVM.java:77)
> 	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> 	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> 	at java.lang.reflect.Method.invoke(Method.java:497)
> 	at sun.rmi.server.UnicastServerRef.dispatch(UnicastServerRef.java:323)
> 	at sun.rmi.transport.Transport$1.run(Transport.java:200)
> 	at sun.rmi.transport.Transport$1.run(Transport.java:197)
> 	at java.security.AccessController.doPrivileged(Native Method)
> 	at sun.rmi.transport.Transport.serviceCall(Transport.java:196)
> 	at sun.rmi.transport.tcp.TCPTransport.handleMessages(TCPTransport.java:568)
> 	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run0(TCPTransport.java:826)
> 	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.lambda$run$256(TCPTransport.java:683)
> 	at java.security.AccessController.doPrivileged(Native Method)
> 	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(TCPTransport.java:682)
> 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> 	at java.lang.Thread.run(Thread.java:745)


--
This message was sent by Atlassian JIRA
(v6.3.4#6332)