Found the bug -- it's a concurrency issue in DirTaxoWriter -- it now
supports addCategory being called concurrently, however not all code paths
support it well. I'll reopen LUCENE-4061 where these changes were made. The
solution shouldn't be too complicated.
Sorry for noise
Shai
On Sat, Jun 9, 2012 at 11:05 AM, <jenkins@sd-datasolutions.de> wrote:
> Build:
> http://jenkins.sd-datasolutions.de/job/Lucene-Solr-4.x-Windows-Java6-64/29/
>
> 1 tests failed.
> REGRESSION:
> org.apache.lucene.facet.taxonomy.directory.TestDirectoryTaxonomyWriter.testConcurrency
>
> Error Message:
> expected:<29364> but was:<29366>
>
> Stack Trace:
> java.lang.AssertionError: expected:<29364> but was:<29366>
> at
> __randomizedtesting.SeedInfo.seed([A7253C6D8EF2FCBC:1D5FEF0EFA11C75]:0)
> at org.junit.Assert.fail(Assert.java:93)
> at org.junit.Assert.failNotEquals(Assert.java:647)
> at org.junit.Assert.assertEquals(Assert.java:128)
> at org.junit.Assert.assertEquals(Assert.java:472)
> at org.junit.Assert.assertEquals(Assert.java:456)
> at
> org.apache.lucene.facet.taxonomy.directory.TestDirectoryTaxonomyWriter.testConcurrency(TestDirectoryTaxonomyWriter.java:253)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at
> com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1969)
> at
> com.carrotsearch.randomizedtesting.RandomizedRunner.access$1100(RandomizedRunner.java:132)
> at
> com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:814)
> at
> com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:875)
> at
> com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:889)
> at
> org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
> at
> org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:32)
> at
> org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
> at
> com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
> at
> org.apache.lucene.util.TestRuleReportUncaughtExceptions$1.evaluate(TestRuleReportUncaughtExceptions.java:68)
> at
> org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
> at
> org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
> at
> com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:821)
> at
> com.carrotsearch.randomizedtesting.RandomizedRunner.access$700(RandomizedRunner.java:132)
> at
> com.carrotsearch.randomizedtesting.RandomizedRunner$3$1.run(RandomizedRunner.java:669)
> at
> com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:695)
> at
> com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:734)
> at
> com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:745)
> at
> org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
> at
> org.apache.lucene.util.TestRuleReportUncaughtExceptions$1.evaluate(TestRuleReportUncaughtExceptions.java:68)
> at
> org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:38)
> at
> org.apache.lucene.util.TestRuleIcuHack$1.evaluate(TestRuleIcuHack.java:51)
> at
> com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
> at
> org.apache.lucene.util.TestRuleNoInstanceHooksOverrides$1.evaluate(TestRuleNoInstanceHooksOverrides.java:53)
> at
> org.apache.lucene.util.TestRuleNoStaticHooksShadowing$1.evaluate(TestRuleNoStaticHooksShadowing.java:52)
> at
> org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:36)
> at
> org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
> at
> org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:56)
> at
> com.carrotsearch.randomizedtesting.RandomizedRunner.runSuite(RandomizedRunner.java:605)
> at
> com.carrotsearch.randomizedtesting.RandomizedRunner.access$400(RandomizedRunner.java:132)
> at
> com.carrotsearch.randomizedtesting.RandomizedRunner$2.run(RandomizedRunner.java:551)
>
>
>
>
> Build Log:
> [...truncated 6769 lines...]
> [junit4] Completed in 0.29s, 4 tests
> [junit4]
> [junit4] Suite: org.apache.lucene.facet.search.DrillDownTest
> [junit4] Completed in 0.06s, 4 tests
> [junit4]
> [junit4] Suite: org.apache.lucene.util.collections.ObjectToFloatMapTest
> [junit4] Completed in 0.06s, 6 tests
> [junit4]
> [junit4] Suite:
> org.apache.lucene.facet.index.CategoryListPayloadStreamTest
> [junit4] Completed in 0.01s, 1 test
> [junit4]
> [junit4] Suite: org.apache.lucene.facet.search.TestCategoryListCache
> [junit4] Completed in 0.37s, 3 tests
> [junit4]
> [junit4] Suite:
> org.apache.lucene.facet.search.TestTopKResultsHandlerRandom
> [junit4] Completed in 13.40s, 1 test
> [junit4]
> [junit4] Suite: org.apache.lucene.facet.search.SamplingWrapperTest
> [junit4] Completed in 11.65s, 1 test
> [junit4]
> [junit4] Suite:
> org.apache.lucene.facet.taxonomy.writercache.cl2o.TestCompactLabelToOrdinal
> [junit4] Completed in 0.99s, 1 test
> [junit4]
> [junit4] Suite: org.apache.lucene.util.collections.IntHashSetTest
> [junit4] Completed in 0.21s, 10 tests
> [junit4]
> [junit4] Suite: org.apache.lucene.facet.search.TestTopKResultsHandler
> [junit4] Completed in 0.68s, 3 tests
> [junit4]
> [junit4] Suite: org.apache.lucene.facet.example.TestMultiCLExample
> [junit4] Completed in 0.11s, 1 test
> [junit4]
> [junit4] Suite:
> org.apache.lucene.facet.search.sampling.SamplingAccumulatorTest
> [junit4] Completed in 8.31s, 1 test
> [junit4]
> [junit4] Suite: org.apache.lucene.facet.search.AdaptiveAccumulatorTest
> [junit4] Completed in 40.07s, 1 test
> [junit4]
> [junit4] Suite: org.apache.lucene.facet.util.TestScoredDocIDsUtils
> [junit4] Completed in 0.19s, 3 tests
> [junit4]
> [junit4] Suite:
> org.apache.lucene.facet.index.categorypolicy.OrdinalPolicyTest
> [junit4] Completed in 0.28s, 2 tests
> [junit4]
> [junit4] Suite:
> org.apache.lucene.facet.taxonomy.directory.TestDirectoryTaxonomyReader
> [junit4] Completed in 0.28s, 7 tests
> [junit4]
> [junit4] Suite:
> org.apache.lucene.facet.taxonomy.directory.TestDirectoryTaxonomyWriter
> [junit4] FAILURE 2.07s | TestDirectoryTaxonomyWriter.testConcurrency
> [junit4] > Throwable #1: java.lang.AssertionError: expected:<29364>
> but was:<29366>
> [junit4] > at
> __randomizedtesting.SeedInfo.seed([A7253C6D8EF2FCBC:1D5FEF0EFA11C75]:0)
> [junit4] > at org.junit.Assert.fail(Assert.java:93)
> [junit4] > at org.junit.Assert.failNotEquals(Assert.java:647)
> [junit4] > at org.junit.Assert.assertEquals(Assert.java:128)
> [junit4] > at org.junit.Assert.assertEquals(Assert.java:472)
> [junit4] > at org.junit.Assert.assertEquals(Assert.java:456)
> [junit4] > at
> org.apache.lucene.facet.taxonomy.directory.TestDirectoryTaxonomyWriter.testConcurrency(TestDirectoryTaxonomyWriter.java:253)
> [junit4] > at
> sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> [junit4] > at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> [junit4] > at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> [junit4] > at java.lang.reflect.Method.invoke(Method.java:597)
> [junit4] > at
> com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1969)
> [junit4] > at
> com.carrotsearch.randomizedtesting.RandomizedRunner.access$1100(RandomizedRunner.java:132)
> [junit4] > at
> com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:814)
> [junit4] > at
> com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:875)
> [junit4] > at
> com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:889)
> [junit4] > at
> org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
> [junit4] > at
> org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:32)
> [junit4] > at
> org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
> [junit4] > at
> com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
> [junit4] > at
> org.apache.lucene.util.TestRuleReportUncaughtExceptions$1.evaluate(TestRuleReportUncaughtExceptions.java:68)
> [junit4] > at
> org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
> [junit4] > at
> org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
> [junit4] > at
> com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:821)
> [junit4] > at
> com.carrotsearch.randomizedtesting.RandomizedRunner.access$700(RandomizedRunner.java:132)
> [junit4] > at
> com.carrotsearch.randomizedtesting.RandomizedRunner$3$1.run(RandomizedRunner.java:669)
> [junit4] > at
> com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:695)
> [junit4] > at
> com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:734)
> [junit4] > at
> com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:745)
> [junit4] > at
> org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
> [junit4] > at
> org.apache.lucene.util.TestRuleReportUncaughtExceptions$1.evaluate(TestRuleReportUncaughtExceptions.java:68)
> [junit4] > at
> org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:38)
> [junit4] > at
> org.apache.lucene.util.TestRuleIcuHack$1.evaluate(TestRuleIcuHack.java:51)
> [junit4] > at
> com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
> [junit4] > at
> org.apache.lucene.util.TestRuleNoInstanceHooksOverrides$1.evaluate(TestRuleNoInstanceHooksOverrides.java:53)
> [junit4] > at
> org.apache.lucene.util.TestRuleNoStaticHooksShadowing$1.evaluate(TestRuleNoStaticHooksShadowing.java:52)
> [junit4] > at
> org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:36)
> [junit4] > at
> org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
> [junit4] > at
> org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:56)
> [junit4] > at
> com.carrotsearch.randomizedtesting.RandomizedRunner.runSuite(RandomizedRunner.java:605)
> [junit4] > at
> com.carrotsearch.randomizedtesting.RandomizedRunner.access$400(RandomizedRunner.java:132)
> [junit4] > at
> com.carrotsearch.randomizedtesting.RandomizedRunner$2.run(RandomizedRunner.java:551)
> [junit4] >
> [junit4] 2> NOTE: reproduce with: ant test
> -Dtestcase=TestDirectoryTaxonomyWriter -Dtests.method=testConcurrency
> -Dtests.seed=A7253C6D8EF2FCBC -Dtests.locale=zh
> -Dtests.timezone=Brazil/DeNoronha -Dargs="-Dfile.encoding=Cp1252"
> [junit4] 2>
> [junit4] > (@AfterClass output)
> [junit4] 2> NOTE: test params are: codec=Lucene40:
> {$full_path$=MockFixedIntBlock(blockSize=444),
> $payloads$=MockFixedIntBlock(blockSize=444)}, sim=DefaultSimilarity,
> locale=zh, timezone=Brazil/DeNoronha
> [junit4] 2> NOTE: Windows 7 6.1 amd64/Sun Microsystems Inc. 1.6.0_32
> (64-bit)/cpus=2,threads=1,free=174573056,total=228786176
> [junit4] 2> NOTE: All tests run in this JVM: [ArrayHashMapTest,
> AssociationsFacetRequestTest, DefaultEnhancementsIndexingParamsTest,
> UnsafeByteArrayOutputStreamTest, CategoryListIteratorTest,
> TestScoredDocIdCollector, CategoryTokenizerTest, TwoEnhancementsTest,
> EnhancementsPayloadIteratorTest, IntToIntMapTest,
> TestMultipleCategoryLists, FloatToObjectMapTest,
> TestTopKInEachNodeResultHandler, ObjectToIntMapTest, TestTaxonomyCombined,
> TestSimpleExample, TestAddTaxonomy, TestCharBlockArray,
> CategoryContainerTest, FacetRequestTest, TestTotalFacetCounts,
> AssociationPropertyTest, TestAdaptiveExample, TestTotalFacetCountsCache,
> FacetSearchParamsTest, EncodingTest, UnsafeByteArrayInputStreamTest,
> FacetsPayloadProcessorProviderTest, CategoryListParamsTest,
> TestAssociationExample, MultiIteratorsPerCLParamsTest, PathPolicyTest,
> IntArrayTest, TestCategoryPath, IntToObjectMapTest,
> TestFacetsAccumulatorWithComplement, Vint8Test, CategoryParentsStreamTest,
> DrillDownTest, ObjectToFloatMapTest, CategoryListPayloadStreamTest,
> TestCategoryListCache, TestTopKResultsHandlerRandom, SamplingWrapperTest,
> TestCompactLabelToOrdinal, IntHashSetTest, TestTopKResultsHandler,
> TestMultiCLExample, SamplingAccumulatorTest, AdaptiveAccumulatorTest,
> TestScoredDocIDsUtils, OrdinalPolicyTest, TestDirectoryTaxonomyReader,
> TestDirectoryTaxonomyWriter]
> [junit4] 2>
> [junit4] Completed in 2.26s, 8 tests, 1 failure <<< FAILURES!
> [...truncated 45 lines...]
>
> [...truncated 6915 lines...]
>
> [...truncated 6915 lines...]
>
> [...truncated 6915 lines...]
>
> [...truncated 6915 lines...]
>
> [...truncated 6915 lines...]
>
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> For additional commands, e-mail: dev-help@lucene.apache.org
>
|