lucenenet-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Laimonas Simutis <lai...@gmail.com>
Subject Re: [TeamCity, FAILED] Build Lucene.NET :: Lucene.NET Core #57
Date Wed, 31 Dec 2014 15:44:03 GMT
I believe this is the culprit:

https://github.com/apache/lucenenet/blob/master/src/Lucene.Net.Tests/core/Index/TestRollingUpdates.cs#L51

If that random get boolean returns true, the default codec is set
to Lucene3xPostingsFormat. That is the codec that throws that exception we
see in the logs.

Any test after TestRollingUpdates that tries to write to an index fails as
they all use Codec.Default. That matches what I see in the output logs
where the failure starts with TestRollingUpdates unit test and then you can
see it happen 1k times after.

Java version is different from the ported .NET version:

https://github.com/apache/lucene-solr/blob/lucene_solr_4_8_0/lucene/core/src/test/org/apache/lucene/index/TestRollingUpdates.java#L45

Instead of setting it to Lucene3xPostingsFormat, it sets it to
MemoryPostingsFormat.
Looking into the fix...



On Wed Dec 31 2014 at 10:02:04 AM Laimonas Simutis <laimis@gmail.com> wrote:

> Itamar,
>
> I doubt that the latest changes is what did it. I just ran a full test
> suite locally on existing master and got the results that match what we
> would expect: 2206 pass, 481 fail.
>
> It is very much likely a case of random tests triggering a certain
> condition that sets default codec to 3x and then when test data is created
> in Test Setup step these exceptions are being triggered. Again, it is a
> guess, but based on what I have observed. Also, keep in mind that I was
> seeing "System.NotSupportedException : this codec can only be used for
> reading" almost from the first full suite runs locally. I just chose to not
> deal with them at that time as it is very inconsistent in how it can be
> reproduced.
>
> My approach right now is to run the full test over and over again with
> some modified logging to see if I can track down how Lucene3x codec ends up
> being used for the tests that are writing data.
>
>
> Laimonas
>
>
> On Wed Dec 31 2014 at 8:45:13 AM Itamar Syn-Hershko <itamar@code972.com>
> wrote:
>
>> IIRC this is due to some confuguration in the tests env not being picked
>> up
>> correctly. Since this only happened since your last change I'd revert it
>> and work from there.
>>
>> --
>>
>> Itamar Syn-Hershko
>> http://code972.com | @synhershko <https://twitter.com/synhershko>
>> Freelance Developer & Consultant
>> Author of RavenDB in Action <http://manning.com/synhershko/>
>>
>> On Wed, Dec 31, 2014 at 1:36 PM, Laimonas Simutis <laimis@gmail.com>
>> wrote:
>>
>> > Looks much worse :)
>> >
>> > The tests are filled with "System.NotSupportedException : this codec can
>> > only be used for reading" failures. I actually ran into this locally
>> almost
>> > all the time whenever I did a full Lucene.Net.Tests solution test run. I
>> > used to wonder why TC tests did not fail in such way. So it could be
>> that
>> > you need to hit the right set of circumstances to make it happen.
>> >
>> > If after encountering these failures I run the tests by fixture or only
>> > certain area (e.g. Lucene.Net.Store), they all pass (the ones that don't
>> > have other issues). My guess is that it is an issue with cleanup
>> somewhere
>> > that is not happening properly cascading to the rest of test cases.
>> >
>> > I will investigate. Please share if anybody else have any clues as to
>> what
>> > is going on.
>> >
>> >
>> > Laimonas
>> >
>> >
>> > On Wed Dec 31 2014 at 4:15:42 AM Itamar Syn-Hershko <itamar@code972.com
>> >
>> > wrote:
>> >
>> > > Ok that doesn't look any better...
>> > >
>> > > --
>> > >
>> > > Itamar Syn-Hershko
>> > > http://code972.com | @synhershko <https://twitter.com/synhershko>
>> > > Freelance Developer & Consultant
>> > > Author of RavenDB in Action <http://manning.com/synhershko/>
>> > >
>> > > On Wed, Dec 31, 2014 at 11:10 AM, <tc@codebetter.com> wrote:
>> > >
>> > > > Build Lucene.NET :: Lucene.NET Core #57 failed (Tests failed: 874
>> (466
>> > > > new), passed: 1396, ignored: 524).
>> > > > Agent: JetBrains-dotnet-agent02
>> > > > Build results:
>> > > > http://teamcity.codebetter.com/viewLog.html?buildId=177083&
>> > > buildTypeId=LuceneNet_Core
>> > > >
>> > > > Failed Tests Summary:
>> > > > Newly failed tests (466 tests, alphabetically ordered, only first
>> 100
>> > > > shown)
>> > > > ==================================================================
>> > > > Lucene.Net.Tests.dll:
>> > > > Lucene.Net.Codecs.Compressing.TestCompressingStoredFieldsFormat.
>> > > BaseStoredFieldsFormatTestCase.TestBinaryFieldOffsetLength
>> > > > Lucene.Net.Tests.dll:
>> > > > Lucene.Net.Codecs.Compressing.TestCompressingTermVectorsForm
>> > > at.TestNoOrds
>> > > > Lucene.Net.Tests.dll:
>> > > > Lucene.Net.Codecs.Lucene40.TestLucene40DocValuesFormat.BaseD
>> > > ocValuesFormatTestCase.TestBytesTwoDocumentsMerged
>> > > > Lucene.Net.Tests.dll:
>> > > > Lucene.Net.Codecs.Lucene40.TestLucene40DocValuesFormat.BaseD
>> > > ocValuesFormatTestCase.TestHugeBinaryValueLimit
>> > > > Lucene.Net.Tests.dll:
>> > > > Lucene.Net.Codecs.Lucene40.TestLucene40DocValuesFormat.BaseD
>> > > ocValuesFormatTestCase.TestSortedBytesTwoDocumentsMerged
>> > > > Lucene.Net.Tests.dll:
>> > > > Lucene.Net.Codecs.Lucene40.TestLucene40PostingsFormat.BasePo
>> > > stingsFormatTestCase.TestDocsAndFreqs
>> > > > Lucene.Net.Tests.dll:
>> > > > Lucene.Net.Codecs.Lucene40.TestLucene40PostingsFormat.BasePo
>> > > stingsFormatTestCase.TestDocsAndFreqsAndPositionsAndOffsetsA
>> ndPayloads
>> > > > Lucene.Net.Tests.dll:
>> > > > Lucene.Net.Codecs.Lucene40.TestLucene40TermVectorsFormat.Bas
>> > > eTermVectorsFormatTestCase.TestMixedOptions
>> > > > Lucene.Net.Tests.dll:
>> > > > Lucene.Net.Codecs.Lucene40.TestLucene40TermVectorsFormat.Bas
>> > > eTermVectorsFormatTestCase.TestRandom
>> > > > Lucene.Net.Tests.dll:
>> > > > Lucene.Net.Codecs.Lucene42.TestLucene42DocValuesFormat.BaseD
>> > > ocValuesFormatTestCase.TestBooleanNumericsVsStoredFields
>> > > > Lucene.Net.Tests.dll:
>> > > > Lucene.Net.Codecs.Lucene42.TestLucene42DocValuesFormat.BaseD
>> > > ocValuesFormatTestCase.TestLongNumericsVsStoredFields
>> > > > Lucene.Net.Tests.dll:
>> > > > Lucene.Net.Codecs.Lucene45.TestLucene45DocValuesFormat.BaseD
>> > > ocValuesFormatTestCase.TestBytesTwoDocumentsMerged
>> > > > Lucene.Net.Tests.dll:
>> > > > Lucene.Net.Codecs.Perfield.TestPerFieldDocValuesFormat.BaseD
>> > > ocValuesFormatTestCase.TestHugeBinaryValueLimit
>> > > > Lucene.Net.Tests.dll:
>> > > > Lucene.Net.Codecs.Perfield.TestPerFieldDocValuesFormat.BaseD
>> > > ocValuesFormatTestCase.TestTwoDocumentsMerged
>> > > > Lucene.Net.Tests.dll:
>> > > > Lucene.Net.Codecs.Perfield.TestPerFieldDocValuesFormat.BaseI
>> > > ndexFileFormatTestCase.TestMergeStability
>> > > > Lucene.Net.Tests.dll:
>> > > > Lucene.Net.Codecs.Perfield.TestPerFieldPostingsFormat.BasePo
>> > > stingsFormatTestCase.TestDocsAndFreqs
>> > > > Lucene.Net.Tests.dll:
>> > > > Lucene.Net.Index.TestAddIndexes.TestAddIndexesWithThreads
>> > > > Lucene.Net.Tests.dll:
>> > > > Lucene.Net.Index.TestBinaryDocValuesUpdates.TestUpdateFewSegments
>> > > > Lucene.Net.Tests.dll:
>> > > > Lucene.Net.Index.TestConsistentFieldNumbers.TestSameFieldNum
>> > > bersAcrossSegments
>> > > > Lucene.Net.Tests.dll:
>> > > > Lucene.Net.Index.TestDocValuesFormat.BaseDocValuesFormatTestCase.
>> > > TestBytesTwoDocumentsMerged
>> > > > Lucene.Net.Tests.dll:
>> > > > Lucene.Net.Index.TestDocValuesFormat.BaseDocValuesFormatTestCase.
>> > > TestSortedSetTwoDocumentsLastMissingMerge
>> > > > Lucene.Net.Tests.dll:
>> > > > Lucene.Net.Index.TestIndexWriterConfig.TestLiveChangeToCFS
>> > > > Lucene.Net.Tests.dll:
>> > > > Lucene.Net.Index.TestIndexWriterForceMerge.TestBackgroundForceMerge
>> > > > Lucene.Net.Tests.dll:
>> > > > Lucene.Net.Index.TestIndexWriterMergePolicy.TestMaxBufferedD
>> ocsChange
>> > > > Lucene.Net.Tests.dll:
>> > > > Lucene.Net.Index.TestIndexWriterReader.TestMergeWarmer
>> > > > Lucene.Net.Tests.dll:
>> > > > Lucene.Net.Index.TestIndexWriterReader.TestReopenAfterNoRealChange
>> > > > Lucene.Net.Tests.dll:
>> > > > Lucene.Net.Index.TestIndexWriterWithThreads.TestIOExceptionD
>> > > uringWriteSegmentWithThreadsOnlyOnce
>> > > > Lucene.Net.Tests.dll:
>> > > > Lucene.Net.Index.TestIndexableField.TestArbitraryFields
>> > > > Lucene.Net.Tests.dll: Lucene.Net.Index.TestMultiDocV
>> alues.TestBinary
>> > > > Lucene.Net.Tests.dll: Lucene.Net.Index.TestMultiDocValues.
>> TestNumerics
>> > > > Lucene.Net.Tests.dll:
>> > > > Lucene.Net.Index.TestNumericDocValuesUpdates.TestSegmentMerges
>> > > > Lucene.Net.Tests.dll: Lucene.Net.Index.TestOmitNorms.TestOmitNorms_
>> Mem
>> > > > Lucene.Net.Tests.dll: Lucene.Net.Index.TestOmitTf.TestNoPrxFile
>> > > > Lucene.Net.Tests.dll:
>> > > > Lucene.Net.Index.TestPostingsFormat.BasePostingsFormatTestCase.
>> > > TestDocsAndFreqsAndPositionsAndOffsetsAndPayloads
>> > > > Lucene.Net.Tests.dll:
>> > > > Lucene.Net.Index.TestPostingsFormat.BasePostingsFormatTestCase.
>> > > TestDocsOnly
>> > > > Lucene.Net.Tests.dll:
>> > > > Lucene.Net.Index.TestRollingUpdates.TestRollingUpdates_Mem
>> > > > Lucene.Net.Tests.dll: Lucene.Net.Index.TestSameToken
>> SamePosition.Test
>> > > > Lucene.Net.Tests.dll:
>> > > > Lucene.Net.Index.TestSameTokenSamePosition.TestMoreDocs
>> > > > Lucene.Net.Tests.dll: Lucene.Net.Index.TestSegmentMerger.Test
>> > > > Lucene.Net.Tests.dll:
>> > Lucene.Net.Index.TestSegmentMerger.TestBuildDocMap
>> > > > Lucene.Net.Tests.dll: Lucene.Net.Index.TestSegmentReader.Test
>> > > > Lucene.Net.Tests.dll:
>> > > > Lucene.Net.Index.TestSegmentReader.TestGetFieldNameVariations
>> > > > Lucene.Net.Tests.dll: Lucene.Net.Index.TestSegmentReader.TestNorms
>> > > > Lucene.Net.Tests.dll:
>> > > > Lucene.Net.Index.TestSegmentReader.TestOutOfBoundsAccess
>> > > > Lucene.Net.Tests.dll:
>> > Lucene.Net.Index.TestSegmentReader.TestTermVectors
>> > > > Lucene.Net.Tests.dll: Lucene.Net.Index.TestSegmentReader.TestTerms
>> > > > Lucene.Net.Tests.dll: Lucene.Net.Index.TestSegmentTermDocs.Test
>> > > > Lucene.Net.Tests.dll: Lucene.Net.Index.TestSegmentTermDocs.
>> TestBadSeek
>> > > > Lucene.Net.Tests.dll: Lucene.Net.Index.TestSegmentTe
>> rmDocs.TestSkipTo
>> > > > Lucene.Net.Tests.dll: Lucene.Net.Index.TestSegmentTermEnum.
>> TestTermEnum
>> > > > Lucene.Net.Tests.dll:
>> > > > Lucene.Net.Index.TestSizeBoundedForceMerge.TestSingleMergeab
>> > > leTooLargeSegment
>> > > > Lucene.Net.Tests.dll:
>> > > >
>> > Lucene.Net.Index.TestSizeBoundedForceMerge.TestSingleNonMerg
>> eableSegment
>> > > > Lucene.Net.Tests.dll:
>> > > > Lucene.Net.Index.TestStoredFieldsFormat.BaseStoredFieldsForm
>> atTestCase.
>> > > TestBulkMergeWithDeletes
>> > > > Lucene.Net.Tests.dll:
>> > > > Lucene.Net.Index.TestStoredFieldsFormat.BaseStoredFieldsForm
>> atTestCase
>> > > .TestConcurrentReads
>> > > > Lucene.Net.Tests.dll:
>> > > > Lucene.Net.Index.TestStoredFieldsFormat.BaseStoredFieldsForm
>> atTestCase
>> > > .TestIndexedBit
>> > > > Lucene.Net.Tests.dll:
>> > > > Lucene.Net.Index.TestStoredFieldsFormat.BaseStoredFieldsForm
>> atTestCase
>> > > .TestReadSkip
>> > > > Lucene.Net.Tests.dll:
>> > > > Lucene.Net.Index.TestStoredFieldsFormat.TestWriteReadMerge
>> > > > Lucene.Net.Tests.dll: Lucene.Net.Index.TestStressIndexing2.
>> > > TestMultiConfig
>> > > > Lucene.Net.Tests.dll:
>> > Lucene.Net.Index.TestSumDocFreq.TestSumDocFreq_Mem
>> > > > Lucene.Net.Tests.dll: Lucene.Net.Index.TestTermVectorsReader.Test
>> > > > Lucene.Net.Tests.dll: Lucene.Net.Index.TestTermVectorsReader.
>> > > TestDocsEnum
>> > > > Lucene.Net.Tests.dll:
>> > > > Lucene.Net.Index.TestTermVectorsReader.TestIllegalIndexableField
>> > > > Lucene.Net.Tests.dll:
>> > > > Lucene.Net.Index.TestTermVectorsReader.TestOffsetReader
>> > > > Lucene.Net.Tests.dll:
>> > > > Lucene.Net.Index.TestTermVectorsReader.TestPositionReader
>> > > > Lucene.Net.Tests.dll: Lucene.Net.Index.TestTermVectorsReader.
>> TestReader
>> > > > Lucene.Net.Tests.dll:
>> > > > Lucene.Net.Index.TestTermVectorsWriter.TestDoubleOffsetCounting
>> > > > Lucene.Net.Tests.dll:
>> > > > Lucene.Net.Index.TestTermVectorsWriter.TestDoubleOffsetCounting2
>> > > > Lucene.Net.Tests.dll:
>> > > >
>> > Lucene.Net.Index.TestTermVectorsWriter.TestEndOffsetPositionCharAnaly
>> zer
>> > > > Lucene.Net.Tests.dll:
>> > > > Lucene.Net.Index.TestTermVectorsWriter.TestEndOffsetPosition
>> Standard
>> > > > Lucene.Net.Tests.dll:
>> > > > Lucene.Net.Index.TestTermVectorsWriter.TestEndOffsetPosition
>> StandardE
>> > > mptyField
>> > > > Lucene.Net.Tests.dll:
>> > > > Lucene.Net.Index.TestTermVectorsWriter.TestEndOffsetPosition
>> StandardE
>> > > mptyField2
>> > > > Lucene.Net.Tests.dll:
>> > > > Lucene.Net.Index.TestTermVectorsWriter.TestEndOffsetPosition
>> StopFilter
>> > > > Lucene.Net.Tests.dll:
>> > > > Lucene.Net.Index.TestTermVectorsWriter.TestEndOffsetPosition
>> WithCachi
>> > > ngTokenFilter
>> > > > Lucene.Net.Tests.dll:
>> > > > Lucene.Net.Index.TestTermVectorsWriter.TestNoTermVectorAfter
>> TermVector
>> > > > Lucene.Net.Tests.dll:
>> > > > Lucene.Net.Index.TestTermVectorsWriter.TestNoTermVectorAfter
>> TermVecto
>> > > rMerge
>> > > > Lucene.Net.Tests.dll:
>> > > > Lucene.Net.Index.TestTermVectorsWriter.TestTermVectorCorruption
>> > > > Lucene.Net.Tests.dll:
>> > > > Lucene.Net.Index.TestTermVectorsWriter.TestTermVectorCorruption2
>> > > > Lucene.Net.Tests.dll:
>> > > > Lucene.Net.Index.TestTermVectorsWriter.TestTermVectorCorruption3
>> > > > Lucene.Net.Tests.dll: Lucene.Net.Index.TestTermsEnum.Test
>> > > > Lucene.Net.Tests.dll: Lucene.Net.Index.TestTermsEnum.TestEasy
>> > > > Lucene.Net.Tests.dll: Lucene.Net.Index.TestTermsEnum.
>> TestIntersectBasic
>> > > > Lucene.Net.Tests.dll:
>> > > > Lucene.Net.Index.TestTermsEnum.TestIntersectEmptyString
>> > > > Lucene.Net.Tests.dll:
>> > Lucene.Net.Index.TestTermsEnum.TestIntersectRandom
>> > > > Lucene.Net.Tests.dll: Lucene.Net.Index.TestTermsEnum.
>> > > TestIntersectStartTerm
>> > > > Lucene.Net.Tests.dll: Lucene.Net.Index.TestTermsEnum.TestZeroTerms
>> > > > Lucene.Net.Tests.dll:
>> > > > Lucene.Net.Index.TestTermsEnum2.TestFiniteVersusInfinite
>> > > > Lucene.Net.Tests.dll: Lucene.Net.Index.TestTermsEnum2.TestIntersect
>> > > > Lucene.Net.Tests.dll: Lucene.Net.Index.TestTermsEnum2.TestSeeking
>> > > > Lucene.Net.Tests.dll: Lucene.Net.Index.TestTermsEnum2.
>> > > TestSeekingAndNexting
>> > > > Lucene.Net.Tests.dll:
>> > > > Lucene.Net.Index.TestTieredMergePolicy.TestForceMergeDeletes
>> > > > Lucene.Net.Tests.dll:
>> > > > Lucene.Net.Index.TestTieredMergePolicy.TestForceMergeDeletes
>> MaxSegSize
>> > > > Lucene.Net.Tests.dll:
>> > > > Lucene.Net.Index.TestTieredMergePolicy.TestIndexWriterDirtSimple
>> > > > Lucene.Net.Tests.dll:
>> > > > Lucene.Net.Index.TestTieredMergePolicy.TestPartialMerge
>> > > > Lucene.Net.Tests.dll:
>> > > > Lucene.Net.Index.TestTransactionRollback.TestRepeatedRollBacks
>> > > > Lucene.Net.Tests.dll:
>> > > > Lucene.Net.Index.TestTransactionRollback.TestRollbackDeletionPolicy
>> > > > Lucene.Net.Tests.dll:
>> > Lucene.Net.Index.TestTryDelete.TestDeleteDocuments
>> > > > Lucene.Net.Tests.dll: Lucene.Net.Index.TestTryDelete.
>> > > TestTryDeleteDocument
>> > > > Lucene.Net.Tests.dll:
>> > > > Lucene.Net.Index.TestTryDelete.TestTryDeleteDocumentCloseAndReopen
>> > > > Lucene.Net.Tests.dll: Lucene.Net.Search.BaseTestRangeFilter
>> > > > Lucene.Net.Tests.dll: Lucene.Net.Search.FuzzyTermOnS
>> hortTermsTest.Test
>> > > >
>> > > > Other failed tests (408 tests, alphabetically ordered, only first
>> 100
>> > > > shown)
>> > > > ==================================================================
>> > > > Changes included (5 changes)
>> > > > ====================================================
>> > > > Change 971b6a3201bf21732ba2c1e56e9e368212e72787 by itamar (0 file):
>> > > > Merge remote-tracking branch 'laimis/unicode_issues'
>> > > >
>> > > >
>> > > > Change 645b21d198523f984c71da307beab445285dd879 by laimis (2
>> files):
>> > > > fixes for handling surrogate chars properly
>> > > >
>> > > >
>> > > > Change 644f10169185efe27b49c11b0a4bbff303005380 by itamar (0 file):
>> > > > Merge remote-tracking branch 'laimis/dictionary_fixes'
>> > > >
>> > > >
>> > > > Change 80e1f9226bcd026abd747de702f85721173cc36d by laimis (1 file):
>> > > > make sure to use offset / length when comparing uft8 buffer
>> > > >
>> > > >
>> > > > Change 0a03242e5f104c55e11a632ea8307e47d05bd0d4 by laimis (2
>> files):
>> > > > use GetTryValue for dictionary lookups
>> > > >
>> > > >
>> > > > see more information about changed files:
>> > > >
>> > http://teamcity.codebetter.com/viewLog.html?tab=buildChanges
>> Div&buildId=
>> > > 177083&buildTypeId=LuceneNet_Core
>> > > >
>> > > >
>> > > > ============================================================
>> > > ================
>> > > > Configure email notifications:
>> > > >
>> > http://teamcity.codebetter.com/profile.html?init=1&tab=userN
>> otifications
>> > > >
>> > >
>> >
>>
>

Mime
  • Unnamed multipart/related (inline, None, 0 bytes)
View raw message