geode-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF subversion and git services (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (GEODE-1558) LuceneQueriesPeerPRDUnitTest.returnCorrectResultsAfterRebalance failed on flushing the queue
Date Thu, 07 Jul 2016 17:09:11 GMT

    [ https://issues.apache.org/jira/browse/GEODE-1558?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15366412#comment-15366412
] 

ASF subversion and git services commented on GEODE-1558:
--------------------------------------------------------

Commit 7b28a8d4e9dc4359d4d11d286f895537864f3a03 in incubator-geode's branch refs/heads/develop
from [~upthewaterspout]
[ https://git-wip-us.apache.org/repos/asf?p=incubator-geode.git;h=7b28a8d ]

GEODE-1558, GEODE-1609: Fixing failures due to rebalancing buckets during lucene index updates

There are two failures we're fixing here:
1) Calling repository.create for create events was leaving duplicate events in the
index because when a bucket fails over, the event is dispatched again on the new primary.
Using the isPossibleDuplicate flag did not work because it was not consistently set to
true for duplicate events. Changed the code to call repository.update even for create events

2) The async event queue was repeatedly dispatching the same events event after
a bucket moved to another node. We changed the async event queue code to filter out
events for buckets that are no longer present on this dispatching member.

Cleaning up the rebalancing test and adding new tests to make these scenarios
more reproducable.

This closes #176


> LuceneQueriesPeerPRDUnitTest.returnCorrectResultsAfterRebalance failed on flushing the
queue
> --------------------------------------------------------------------------------------------
>
>                 Key: GEODE-1558
>                 URL: https://issues.apache.org/jira/browse/GEODE-1558
>             Project: Geode
>          Issue Type: Bug
>          Components: lucene
>            Reporter: Aparna Dharmakkan
>
> The error was:
> {noformat}
> java.lang.AssertionError
> 	at org.junit.Assert.fail(Assert.java:86)
> 	at org.junit.Assert.assertTrue(Assert.java:41)
> 	at org.junit.Assert.assertTrue(Assert.java:52)
> 	at com.gemstone.gemfire.cache.lucene.LuceneQueriesPRBase.returnCorrectResultsAfterRebalance(LuceneQueriesPRBase.java:64)
> 	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> 	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> 	at java.lang.reflect.Method.invoke(Method.java:498)
> 	at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
> 	at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> 	at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
> 	at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> 	at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
> 	at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
> 	at org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:55)
> 	at org.junit.rules.RunRules.evaluate(RunRules.java:20)
> 	at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
> 	at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
> 	at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
> 	at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
> 	at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
> 	at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
> 	at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
> 	at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
> 	at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
> 	at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
> 	at org.gradle.api.internal.tasks.testing.junit.JUnitTestClassExecuter.runTestClass(JUnitTestClassExecuter.java:112)
> 	at org.gradle.api.internal.tasks.testing.junit.JUnitTestClassExecuter.execute(JUnitTestClassExecuter.java:56)
> 	at org.gradle.api.internal.tasks.testing.junit.JUnitTestClassProcessor.processTestClass(JUnitTestClassProcessor.java:66)
> 	at org.gradle.api.internal.tasks.testing.SuiteTestClassProcessor.processTestClass(SuiteTestClassProcessor.java:51)
> 	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> 	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> 	at java.lang.reflect.Method.invoke(Method.java:498)
> 	at org.gradle.messaging.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:35)
> 	at org.gradle.messaging.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:24)
> 	at org.gradle.messaging.dispatch.ContextClassLoaderDispatch.dispatch(ContextClassLoaderDispatch.java:32)
> 	at org.gradle.messaging.dispatch.ProxyDispatchAdapter$DispatchingInvocationHandler.invoke(ProxyDispatchAdapter.java:93)
> 	at com.sun.proxy.$Proxy2.processTestClass(Unknown Source)
> 	at org.gradle.api.internal.tasks.testing.worker.TestWorker.processTestClass(TestWorker.java:109)
> 	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> 	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> 	at java.lang.reflect.Method.invoke(Method.java:498)
> 	at org.gradle.messaging.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:35)
> 	at org.gradle.messaging.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:24)
> 	at org.gradle.messaging.remote.internal.hub.MessageHub$Handler.run(MessageHub.java:360)
> 	at org.gradle.internal.concurrent.ExecutorPolicy$CatchAndRecordFailures.onExecute(ExecutorPolicy.java:54)
> 	at org.gradle.internal.concurrent.StoppableExecutorImpl$1.run(StoppableExecutorImpl.java:40)
> 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> 	at java.lang.Thread.run(Thread.java:745)
> {noformat}
> and the log output:
> {noformat}
> [vm_0]49.448: [GC (Allocation Failure) [PSYoungGen: 150556K->11769K(162816K)] 165672K->37374K(230400K),
0.0693682 secs] [Times: user=0.06 sys=0.01, real=0.07 secs] 
> [vm_3][warn 2016/06/16 16:08:16.915 PDT <Event Processor for GatewaySender_AsyncEventQueue_index#_region_0>
tid=0x48] com.gemstone.gemfire.internal.cache.wan.GatewaySenderEventCallbackDispatcher@1215f139:
Exception during processing batch 0
> [vm_3]com.gemstone.gemfire.internal.cache.wan.GatewaySenderException: com.gemstone.gemfire.internal.cache.wan.GatewaySenderEventCallbackDispatcher@1215f139:
Exception during processing batch 0, caused by com.gemstone.gemfire.internal.cache.PrimaryBucketException:
Bucket _B__index#__region.files_0 is not primary. Current primary holder is 192.168.1.17(12468)<ec><v51>:1027
> [vm_3]	at com.gemstone.gemfire.internal.cache.wan.GatewaySenderEventCallbackDispatcher.dispatchBatch(GatewaySenderEventCallbackDispatcher.java:177)
> [vm_3]	at com.gemstone.gemfire.internal.cache.wan.GatewaySenderEventCallbackDispatcher.dispatchBatch(GatewaySenderEventCallbackDispatcher.java:87)
> [vm_3]	at com.gemstone.gemfire.internal.cache.wan.AbstractGatewaySenderEventProcessor.processQueue(AbstractGatewaySenderEventProcessor.java:613)
> [vm_3]	at com.gemstone.gemfire.internal.cache.wan.AbstractGatewaySenderEventProcessor.run(AbstractGatewaySenderEventProcessor.java:1058)
> [vm_3]Caused by: com.gemstone.gemfire.internal.cache.PrimaryBucketException: Bucket _B__index#__region.files_0
is not primary. Current primary holder is 192.168.1.17(12468)<ec><v51>:1027
> [vm_3]	at com.gemstone.gemfire.internal.cache.BucketRegion.checkForPrimary(BucketRegion.java:670)
> [vm_3]	at com.gemstone.gemfire.internal.cache.BucketRegion.doLockForPrimary(BucketRegion.java:721)
> [vm_3]	at com.gemstone.gemfire.internal.cache.BucketRegion.beginLocalWrite(BucketRegion.java:697)
> [vm_3]	at com.gemstone.gemfire.internal.cache.BucketRegion.virtualPut(BucketRegion.java:476)
> [vm_3]	at com.gemstone.gemfire.internal.cache.LocalRegionDataView.putEntry(LocalRegionDataView.java:132)
> [vm_3]	at com.gemstone.gemfire.internal.cache.LocalRegion.basicPut(LocalRegion.java:5241)
> [vm_3]	at com.gemstone.gemfire.internal.cache.LocalRegion.putIfAbsent(LocalRegion.java:12022)
> [vm_3]	at com.gemstone.gemfire.internal.cache.LocalRegion.putIfAbsent(LocalRegion.java:12046)
> [vm_3]	at com.gemstone.gemfire.cache.lucene.internal.filesystem.FileSystem.createFile(FileSystem.java:67)
> [vm_3]	at com.gemstone.gemfire.cache.lucene.internal.directory.RegionDirectory.createOutput(RegionDirectory.java:84)
> [vm_3]	at org.apache.lucene.store.LockValidatingDirectoryWrapper.createOutput(LockValidatingDirectoryWrapper.java:44)
> [vm_3]	at org.apache.lucene.store.TrackingDirectoryWrapper.createOutput(TrackingDirectoryWrapper.java:43)
> [vm_3]	at org.apache.lucene.codecs.compressing.CompressingStoredFieldsWriter.<init>(CompressingStoredFieldsWriter.java:108)
> [vm_3]	at org.apache.lucene.codecs.compressing.CompressingStoredFieldsFormat.fieldsWriter(CompressingStoredFieldsFormat.java:128)
> [vm_3]	at org.apache.lucene.codecs.lucene50.Lucene50StoredFieldsFormat.fieldsWriter(Lucene50StoredFieldsFormat.java:183)
> [vm_3]	at org.apache.lucene.index.DefaultIndexingChain.initStoredFieldsWriter(DefaultIndexingChain.java:83)
> [vm_3]	at org.apache.lucene.index.DefaultIndexingChain.startStoredFields(DefaultIndexingChain.java:331)
> [vm_3]	at org.apache.lucene.index.DefaultIndexingChain.processDocument(DefaultIndexingChain.java:368)
> [vm_3]	at org.apache.lucene.index.DocumentsWriterPerThread.updateDocument(DocumentsWriterPerThread.java:232)
> [vm_3]	at org.apache.lucene.index.DocumentsWriter.updateDocument(DocumentsWriter.java:449)
> [vm_3]	at org.apache.lucene.index.IndexWriter.updateDocument(IndexWriter.java:1492)
> [vm_3]	at org.apache.lucene.index.IndexWriter.addDocument(IndexWriter.java:1271)
> [vm_3]	at com.gemstone.gemfire.cache.lucene.internal.repository.IndexRepositoryImpl.create(IndexRepositoryImpl.java:70)
> [vm_3]	at com.gemstone.gemfire.cache.lucene.internal.LuceneEventListener.processEvents(LuceneEventListener.java:78)
> [vm_3]	at com.gemstone.gemfire.internal.cache.wan.GatewaySenderEventCallbackDispatcher.dispatchBatch(GatewaySenderEventCallbackDispatcher.java:168)
> [vm_3]	... 3 more
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message