lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Shai Erera (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (LUCENE-4402) TestAddTaxonomy.testConcurrency failure
Date Wed, 19 Sep 2012 11:59:07 GMT

    [ https://issues.apache.org/jira/browse/LUCENE-4402?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13458607#comment-13458607
] 

Shai Erera commented on LUCENE-4402:
------------------------------------

Found the problem. On 3.6, all DirTW methods are synchronized, but addTaxonomy wasn't. The
idea was to let someone call addTaxo in parallel to addCategory (hence the test), while addTaxo
itself calls addCategory internally (which is synchronized). The problem is that prior to
calling addCategory, it calls findCategory, which is not synchronized and cache implementations
are not thread safe (in 3.6).

So the solution is to call addCategory from addTaxonomy in 3x, 4x and trunk. In 4x and trunk
addCategory is not synchronized, checking the cache first, so the existing code is redundant.
In 3x since we cannot trust findCategory in a multi-threaded env., calling addCategory will
solve it.

I'll modify the code and commit the changes - a patch is trivial therefore I won't post one.
                
> TestAddTaxonomy.testConcurrency failure
> ---------------------------------------
>
>                 Key: LUCENE-4402
>                 URL: https://issues.apache.org/jira/browse/LUCENE-4402
>             Project: Lucene - Core
>          Issue Type: Bug
>          Components: modules/facet
>    Affects Versions: 3.6.2
>            Reporter: Robert Muir
>
> on the 3.x branch:
> {noformat}
>     [junit] Testsuite: org.apache.lucene.facet.taxonomy.directory.TestAddTaxonomy
>     [junit] Testcase: testConcurrency(org.apache.lucene.facet.taxonomy.directory.TestAddTaxonomy):
Caused an ERROR
>     [junit] Index: 1, Size: 2
>     [junit] java.lang.IndexOutOfBoundsException: Index: 1, Size: 2
>     [junit] 	at java.util.ArrayList.rangeCheck(ArrayList.java:604)
>     [junit] 	at java.util.ArrayList.get(ArrayList.java:382)
>     [junit] 	at org.apache.lucene.facet.taxonomy.writercache.cl2o.CharBlockArray.charAt(CharBlockArray.java:148)
>     [junit] 	at org.apache.lucene.facet.taxonomy.CategoryPath.equalsToSerialized(CategoryPath.java:888)
>     [junit] 	at org.apache.lucene.facet.taxonomy.writercache.cl2o.CompactLabelToOrdinal.getOrdinal(CompactLabelToOrdinal.java:323)
>     [junit] 	at org.apache.lucene.facet.taxonomy.writercache.cl2o.CompactLabelToOrdinal.getOrdinal(CompactLabelToOrdinal.java:163)
>     [junit] 	at org.apache.lucene.facet.taxonomy.writercache.cl2o.Cl2oTaxonomyWriterCache.get(Cl2oTaxonomyWriterCache.java:49)
>     [junit] 	at org.apache.lucene.facet.taxonomy.directory.DirectoryTaxonomyWriter.findCategory(DirectoryTaxonomyWriter.java:386)
>     [junit] 	at org.apache.lucene.facet.taxonomy.directory.DirectoryTaxonomyWriter.addTaxonomy(DirectoryTaxonomyWriter.java:833)
>     [junit] 	at org.apache.lucene.facet.taxonomy.directory.TestAddTaxonomy.testConcurrency(TestAddTaxonomy.java:206)
>     [junit] 	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>     [junit] 	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
>     [junit] 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>     [junit] 	at java.lang.reflect.Method.invoke(Method.java:601)
>     [junit] 	at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:45)
>     [junit] 	at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)
>     [junit] 	at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:42)
>     [junit] 	at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:20)
>     [junit] 	at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:28)
>     [junit] 	at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:30)
>     [junit] 	at org.apache.lucene.util.LuceneTestCase$SubclassSetupTeardownRule$1.evaluate(LuceneTestCase.java:630)
>     [junit] 	at org.apache.lucene.util.LuceneTestCase$InternalSetupTeardownRule$1.evaluate(LuceneTestCase.java:536)
>     [junit] 	at org.apache.lucene.util.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:67)
>     [junit] 	at org.apache.lucene.util.LuceneTestCase$TestResultInterceptorRule$1.evaluate(LuceneTestCase.java:457)
>     [junit] 	at org.apache.lucene.util.UncaughtExceptionsRule$1.evaluate(UncaughtExceptionsRule.java:74)
>     [junit] 	at org.apache.lucene.util.LuceneTestCase$SaveThreadAndTestNameRule$1.evaluate(LuceneTestCase.java:508)
>     [junit] 	at org.junit.rules.RunRules.evaluate(RunRules.java:18)
>     [junit] 	at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:263)
>     [junit] 	at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:68)
>     [junit] 	at org.apache.lucene.util.LuceneTestCaseRunner.runChild(LuceneTestCaseRunner.java:146)
>     [junit] 	at org.apache.lucene.util.LuceneTestCaseRunner.runChild(LuceneTestCaseRunner.java:50)
>     [junit] 	at org.junit.runners.ParentRunner$3.run(ParentRunner.java:231)
>     [junit] 	at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:60)
>     [junit] 	at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:229)
>     [junit] 	at org.junit.runners.ParentRunner.access$000(ParentRunner.java:50)
>     [junit] 	at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:222)
>     [junit] 	at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:28)
>     [junit] 	at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:30)
>     [junit] 	at org.apache.lucene.util.UncaughtExceptionsRule$1.evaluate(UncaughtExceptionsRule.java:74)
>     [junit] 	at org.apache.lucene.util.StoreClassNameRule$1.evaluate(StoreClassNameRule.java:36)
>     [junit] 	at org.apache.lucene.util.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:67)
>     [junit] 	at org.junit.rules.RunRules.evaluate(RunRules.java:18)
>     [junit] 	at org.junit.runners.ParentRunner.run(ParentRunner.java:300)
>     [junit] 	at junit.framework.JUnit4TestAdapter.run(JUnit4TestAdapter.java:39)
>     [junit] 	at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.run(JUnitTestRunner.java:518)
>     [junit] 	at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.launch(JUnitTestRunner.java:1052)
>     [junit] 	at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.main(JUnitTestRunner.java:879)
>     [junit] 
>     [junit] 
>     [junit] Tests run: 5, Failures: 0, Errors: 1, Time elapsed: 0.342 sec
>     [junit] 
>     [junit] ------------- Standard Error -----------------
>     [junit] NOTE: Ignoring nightly-only test method 'testBig'
>     [junit] WARNING: test method: 'testConcurrency' left thread running: Thread[Thread-1,5,main]
>     [junit] RESOURCE LEAK: test method: 'testConcurrency' left 1 thread(s) running
>     [junit] NOTE: reproduce with: ant test -Dtestcase=TestAddTaxonomy -Dtestmethod=testConcurrency
-Dtests.seed=ad99a0aac3d5bf3:4af538a36d0c94b4:-7b609955992bc1c9 -Dargs="-Dfile.encoding=UTF-8"
>     [junit] NOTE: reproduce with: ant test -Dtestcase=TestAddTaxonomy -Dtestmethod=testConcurrency
-Dtests.seed=ad99a0aac3d5bf3:4af538a36d0c94b4:-7b609955992bc1c9 -Dargs="-Dfile.encoding=UTF-8"
>     [junit] NOTE: test params are: locale=mt_MT, timezone=Etc/GMT+6
>     [junit] NOTE: all tests run in this JVM:
>     [junit] [TwoEnhancementsTest, TestAdaptiveExample, CategoryListPayloadStreamTest,
CategoryAttributeImplTest, PerDimensionIndexingParamsTest, AdaptiveAccumulatorTest, TestFacetArrays,
TestFacetsAccumulatorWithComplement, FacetSearchParamsTest, SamplingAccumulatorTest, TestTaxonomyCombined,
TestAddTaxonomy]
>     [junit] NOTE: Linux 3.2.0-24-generic amd64/Oracle Corporation 1.7.0_01 (64-bit)/cpus=8,threads=1,free=171017984,total=269352960
>     [junit] ------------- ---------------- ---------------
>     [junit] TEST org.apache.lucene.facet.taxonomy.directory.TestAddTaxonomy FAILED
> {noformat}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org


Mime
View raw message