lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Michael McCandless <luc...@mikemccandless.com>
Subject Re: [JENKINS] Lucene-Solr-tests-only-4.x-java7 - Build # 228 - Still Failing
Date Tue, 31 Jul 2012 10:47:33 GMT
Thanks Robert, I'll dig.

Separately: how come we didn't get a heap dump from the failure...?

Mike McCandless

http://blog.mikemccandless.com

On Mon, Jul 30, 2012 at 10:15 PM, Robert Muir <rcmuir@gmail.com> wrote:
> This seed OOMs here too.
>
> I added these prints:
>     System.out.println(RamUsageEstimator.sizeOf(fields));
>     System.out.println(RamUsageEstimator.sizeOf(fieldsLive));
>
> I see:
> [junit4:junit4]   1> 367701000
> [junit4:junit4]   1> 222881224
>
> (I am sure they are likely sharing some ram). But this is a massive
> amount of ram for this test.
>
> I will temporarily disable the multiplier, until the test can be made
> more ram efficient. it grows linearly with the multiplier, i think
> because of all the lists within lists within maps etc :)
>
> [junit4:junit4]   1> 136660616
> [junit4:junit4]   1> 89595032
>
>
> On Mon, Jul 30, 2012 at 8:30 PM, Apache Jenkins Server
> <jenkins@builds.apache.org> wrote:
>> Build: https://builds.apache.org/job/Lucene-Solr-tests-only-4.x-java7/228/
>>
>> 1 tests failed.
>> REGRESSION:  org.apache.lucene.index.TestPostingsFormat.test
>>
>> Error Message:
>> Java heap space
>>
>> Stack Trace:
>> java.lang.OutOfMemoryError: Java heap space
>>         at __randomizedtesting.SeedInfo.seed([E8AAFF6FA7F41694:60FEC0B509087B6C]:0)
>>         at org.apache.lucene.util.BytesRef.<init>(BytesRef.java:75)
>>         at org.apache.lucene.util.fst.ByteSequenceOutputs.read(ByteSequenceOutputs.java:124)
>>         at org.apache.lucene.util.fst.ByteSequenceOutputs.read(ByteSequenceOutputs.java:33)
>>         at org.apache.lucene.util.fst.FST.readNextRealArc(FST.java:960)
>>         at org.apache.lucene.util.fst.NodeHash.hash(NodeHash.java:106)
>>         at org.apache.lucene.util.fst.NodeHash.add(NodeHash.java:124)
>>         at org.apache.lucene.util.fst.Builder.compileNode(Builder.java:195)
>>         at org.apache.lucene.util.fst.Builder.freezeTail(Builder.java:287)
>>         at org.apache.lucene.util.fst.Builder.add(Builder.java:392)
>>         at org.apache.lucene.codecs.memory.MemoryPostingsFormat$TermsWriter.finishTerm(MemoryPostingsFormat.java:257)
>>         at org.apache.lucene.index.TestPostingsFormat.buildIndex(TestPostingsFormat.java:441)
>>         at org.apache.lucene.index.TestPostingsFormat.test(TestPostingsFormat.java:896)
>>         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>>         at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
>>         at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>>         at java.lang.reflect.Method.invoke(Method.java:601)
>>         at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1995)
>>         at com.carrotsearch.randomizedtesting.RandomizedRunner.access$1100(RandomizedRunner.java:132)
>>         at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:818)
>>         at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:877)
>>         at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:891)
>>         at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
>>         at org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:32)
>>         at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
>>         at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
>>         at org.apache.lucene.util.TestRuleReportUncaughtExceptions$1.evaluate(TestRuleReportUncaughtExceptions.java:68)
>>         at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
>>         at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
>>         at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
>>         at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:825)
>>         at com.carrotsearch.randomizedtesting.RandomizedRunner.access$700(RandomizedRunner.java:132)
>>         at com.carrotsearch.randomizedtesting.RandomizedRunner$3$1.run(RandomizedRunner.java:671)
>>
>>
>>
>>
>> Build Log:
>> [...truncated 1197 lines...]
>> [junit4:junit4] Suite: org.apache.lucene.index.TestPostingsFormat
>> [junit4:junit4] ERROR   4.61s J0 | TestPostingsFormat.test
>> [junit4:junit4]    > Throwable #1: java.lang.OutOfMemoryError: Java heap space
>> [junit4:junit4]    >    at __randomizedtesting.SeedInfo.seed([E8AAFF6FA7F41694:60FEC0B509087B6C]:0)
>> [junit4:junit4]    >    at org.apache.lucene.util.BytesRef.<init>(BytesRef.java:75)
>> [junit4:junit4]    >    at org.apache.lucene.util.fst.ByteSequenceOutputs.read(ByteSequenceOutputs.java:124)
>> [junit4:junit4]    >    at org.apache.lucene.util.fst.ByteSequenceOutputs.read(ByteSequenceOutputs.java:33)
>> [junit4:junit4]    >    at org.apache.lucene.util.fst.FST.readNextRealArc(FST.java:960)
>> [junit4:junit4]    >    at org.apache.lucene.util.fst.NodeHash.hash(NodeHash.java:106)
>> [junit4:junit4]    >    at org.apache.lucene.util.fst.NodeHash.add(NodeHash.java:124)
>> [junit4:junit4]    >    at org.apache.lucene.util.fst.Builder.compileNode(Builder.java:195)
>> [junit4:junit4]    >    at org.apache.lucene.util.fst.Builder.freezeTail(Builder.java:287)
>> [junit4:junit4]    >    at org.apache.lucene.util.fst.Builder.add(Builder.java:392)
>> [junit4:junit4]    >    at org.apache.lucene.codecs.memory.MemoryPostingsFormat$TermsWriter.finishTerm(MemoryPostingsFormat.java:257)
>> [junit4:junit4]    >    at org.apache.lucene.index.TestPostingsFormat.buildIndex(TestPostingsFormat.java:441)
>> [junit4:junit4]    >    at org.apache.lucene.index.TestPostingsFormat.test(TestPostingsFormat.java:896)
>> [junit4:junit4]    >    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native
Method)
>> [junit4:junit4]    >    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
>> [junit4:junit4]    >    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>> [junit4:junit4]    >    at java.lang.reflect.Method.invoke(Method.java:601)
>> [junit4:junit4]    >    at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1995)
>> [junit4:junit4]    >    at com.carrotsearch.randomizedtesting.RandomizedRunner.access$1100(RandomizedRunner.java:132)
>> [junit4:junit4]    >    at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:818)
>> [junit4:junit4]    >    at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:877)
>> [junit4:junit4]    >    at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:891)
>> [junit4:junit4]    >    at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
>> [junit4:junit4]    >    at org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:32)
>> [junit4:junit4]    >    at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
>> [junit4:junit4]    >    at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
>> [junit4:junit4]    >    at org.apache.lucene.util.TestRuleReportUncaughtExceptions$1.evaluate(TestRuleReportUncaughtExceptions.java:68)
>> [junit4:junit4]    >    at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
>> [junit4:junit4]    >    at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
>> [junit4:junit4]    >    at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
>> [junit4:junit4]    >    at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:825)
>> [junit4:junit4]    >    at com.carrotsearch.randomizedtesting.RandomizedRunner.access$700(RandomizedRunner.java:132)
>> [junit4:junit4]    >    at com.carrotsearch.randomizedtesting.RandomizedRunner$3$1.run(RandomizedRunner.java:671)
>> [junit4:junit4]    >
>> [junit4:junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=TestPostingsFormat
-Dtests.method=test -Dtests.seed=E8AAFF6FA7F41694 -Dtests.multiplier=3 -Dtests.slow=true -Dtests.locale=ja
-Dtests.timezone=America/Fort_Wayne -Dtests.file.encoding=US-ASCII
>> [junit4:junit4]   2>
>> [junit4:junit4]    > (@AfterClass output)
>> [junit4:junit4]   2> NOTE: test params are: codec=Lucene40: {=PostingsFormat(name=MockSep),
rl=Lucene40(minBlockSize=82 maxBlockSize=257), xgeplm=PostingsFormat(name=Memory doPackFST=
false), kojsjftf=PostingsFormat(name=NestedPulsing)}, sim=DefaultSimilarity, locale=ja, timezone=America/Fort_Wayne
>> [junit4:junit4]   2> NOTE: FreeBSD 9.0-RELEASE amd64/Oracle Corporation 1.7.0_04
(64-bit)/cpus=16,threads=1,free=33223808,total=536870912
>> [junit4:junit4]   2> NOTE: All tests run in this JVM: [TestSearchAfter, TestDocValuesIndexing,
TestWeakIdentityMap, TestSimpleExplanationsOfNonMatches, TestFilteredQuery, TestFieldCacheTermsFilter,
Nested, Nested, Nested, Nested, Nested, Nested, Nested, Nested, Nested, Nested, Nested, Nested,
Nested, Nested, Nested, Nested, Nested, Nested, Nested, Nested, Nested, TestSpanExplanations,
TestIndexWriterWithThreads, TestMaxTermFrequency, TestStressIndexing2, TestIndexableBinaryStringTools,
TestCheckIndex, TestCharsRef, TestTermdocPerf, TestDateSort, TestSetOnce, TestMixedCodecs,
TestSloppyPhraseQuery2, TestMultiFields, TestSnapshotDeletionPolicy, TestTimeLimitingCollector,
TestNoDeletionPolicy, TestLazyProxSkipping, Test2BTerms, TestIndexWriterForceMerge, TestRollingUpdates,
TestTermVectorsReader, TestAssertions, TestSpecialOperations, TestFilterAtomicReader, TestElevationComparator,
TestMultiMMap, TestByteSlices, TestPayloadTermQuery, TestDocumentWriter, TestSizeBoundedForceMerge,
TestFuzzyQuery, TestDeletionPolicy, TestShardSearching, TestNumericUtils, TestSpanSearchEquivalence,
TestDisjunctionMaxQuery, TestByteArrayDataInput, TestMultiPhraseQuery, TestTopDocsCollector,
TestFixedBitSet, TestSearch, TestTransactionRollback, TestTermVectors, TestIndexWriter, TestDirectoryReaderReopen,
TestOmitTf, TestCopyBytes, TestRegexpQuery, TestSmallFloat, TestIndexWriterNRTIsCurrent, TestMockCharFilter,
TestPrefixCodedTerms, TestLevenshteinAutomata, Nested, Nested, TestMultiTermConstantScore,
TestIndexWriterReader, TestWindowsMMap, TestDemo, TestParallelCompositeReader, TestScoreCachingWrappingScorer,
TestFileSwitchDirectory, TestArrayUtil, TestPagedBytes, TestIndexWriterLockRelease, InBeforeClass,
InAfterClass, InTestMethod, NonStringProperties, TestBasicOperations, TestSpanExplanationsOfNonMatches,
TestSortedVIntList, TestScorerPerf, TestPrefixInBooleanQuery, TestRegexpRandom, TestNorms,
TestIndexWriterExceptions, TestBasics, TestDuelingCodecs, TestIndexWriterMergePolicy, TestNeverDelete,
TestFlushByRamOrCountsPolicy, TestTopDocsMerge, TestCollectionUtil, TestCompoundFile, TestSimpleExplanations,
TestCustomNorms, TestStressIndexing, TestDeterminizeLexicon, TestDocumentsWriterStallControl,
TestRollingCharBuffer, TestRegexpRandom2, TestConsistentFieldNumbers, TestDoubleBarrelLRUCache,
TestCrashCausesCorruptIndex, TestDocValues, TestUTF32ToUTF8, TestTransactions, TestThreadedForceMerge,
TestSegmentReader, TestPerFieldPostingsFormat, TestFieldCache, TestPayloadNearQuery, TestPrefixRandom,
TestDocsAndPositions, TestFlex, TestIndexableField, TestSimilarity2, TestAllFilesHaveCodecHeader,
TestMultiLevelSkipList, TestExternalCodecs, Nested2, Nested3, Nested1, TestTermRangeQuery,
TestSpansAdvanced2, TestSpanMultiTermQueryWrapper, TestComplexExplanationsOfNonMatches, TestSumDocFreq,
TestPerSegmentDeletes, TestParallelReaderEmptyIndex, TestIndexFileDeleter, Before3, Before3,
TestDocValuesScoring, TestDocIdSet, TestFieldValueFilter, TestAppendingCodec, TestSpansAdvanced,
TestConstantScoreQuery, TestPositionIncrement, Nested, TestReaderClosed, TestNot, TestAutomatonQueryUnicode,
TestIndexWriterOnJRECrash, TestBinaryDocument, TestSpanFirstQuery, TestPriorityQueue, TestDocBoost,
TestIntBlockCodec, TestIsCurrent, MultiCollectorTest, TestPrefixFilter, TestVersion, TestNoMergeScheduler,
TestCachingTokenFilter, TestIOUtils, TestCharFilter, TestPostingsFormat]
>> [junit4:junit4]   2>
>> [junit4:junit4] Completed on J0 in 13.67s, 1 test, 1 error <<< FAILURES!
>>
>> [...truncated 5 lines...]
>> BUILD FAILED
>> /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-tests-only-4.x-java7/checkout/build.xml:29:
The following error occurred while executing this line:
>> /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-tests-only-4.x-java7/checkout/lucene/build.xml:50:
The following error occurred while executing this line:
>> /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-tests-only-4.x-java7/checkout/lucene/common-build.xml:1095:
The following error occurred while executing this line:
>> /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-tests-only-4.x-java7/checkout/lucene/common-build.xml:790:
There were test failures: 309 suites, 1736 tests, 1 error, 14 ignored (12 assumptions)
>>
>> Total time: 6 minutes 14 seconds
>> Build step 'Execute shell' marked build as failure
>> Archiving artifacts
>> Recording test results
>> Email was triggered for: Failure
>> Sending email for trigger: Failure
>>
>>
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
>> For additional commands, e-mail: dev-help@lucene.apache.org
>
>
>
> --
> lucidimagination.com
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> For additional commands, e-mail: dev-help@lucene.apache.org
>

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org


Mime
View raw message