lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Simon Willnauer (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (LUCENE-2984) Move hasVectors() & hasProx() responsibility out of SegmentInfo to FieldInfos
Date Wed, 11 May 2011 16:17:48 GMT

     [ https://issues.apache.org/jira/browse/LUCENE-2984?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Simon Willnauer updated LUCENE-2984:
------------------------------------

    Attachment: LUCENE-2984.patch

new patch!
I was running tests with the previous patch and tripped a very nifty exception.
{noformat}
 [junit] Testsuite: org.apache.lucene.store.TestLockFactory
    [junit] Testcase: testStressLocksNativeFSLockFactory(org.apache.lucene.store.TestLockFactory):
FAILED
    [junit] IndexWriter hit unexpected exceptions
    [junit] junit.framework.AssertionFailedError: IndexWriter hit unexpected exceptions
    [junit] 	at org.apache.lucene.store.TestLockFactory._testStressLocks(TestLockFactory.java:164)
    [junit] 	at org.apache.lucene.store.TestLockFactory.testStressLocksNativeFSLockFactory(TestLockFactory.java:144)
    [junit] 	at org.apache.lucene.util.LuceneTestCase$LuceneTestCaseRunner.runChild(LuceneTestCase.java:1282)
    [junit] 	at org.apache.lucene.util.LuceneTestCase$LuceneTestCaseRunner.runChild(LuceneTestCase.java:1211)
    [junit] 
    [junit] 
    [junit] Tests run: 11, Failures: 1, Errors: 0, Time elapsed: 7.092 sec
    [junit] 
    [junit] ------------- Standard Output ---------------
    [junit] Stress Test Index Writer: creation hit unexpected IOException: java.io.FileNotFoundException:
_u.fnm
    [junit] java.io.FileNotFoundException: _u.fnm
    [junit] 	at org.apache.lucene.store.MockDirectoryWrapper.openInput(MockDirectoryWrapper.java:386)
    [junit] 	at org.apache.lucene.index.FieldInfos.<init>(FieldInfos.java:273)
    [junit] 	at org.apache.lucene.index.SegmentInfo.loadFieldInfos(SegmentInfo.java:264)
    [junit] 	at org.apache.lucene.index.SegmentInfo.getFieldInfos(SegmentInfo.java:315)
    [junit] 	at org.apache.lucene.index.SegmentInfo.files(SegmentInfo.java:603)
    [junit] 	at org.apache.lucene.index.SegmentInfos.files(SegmentInfos.java:873)
    [junit] 	at org.apache.lucene.index.IndexFileDeleter$CommitPoint.<init>(IndexFileDeleter.java:625)
    [junit] 	at org.apache.lucene.index.IndexFileDeleter.<init>(IndexFileDeleter.java:199)
    [junit] 	at org.apache.lucene.index.IndexWriter.<init>(IndexWriter.java:830)
    [junit] 	at org.apache.lucene.store.TestLockFactory$WriterThread.run(TestLockFactory.java:283)
    [junit] Stress Test Index Writer: creation hit unexpected IOException: java.io.FileNotFoundException:
_u.fnm
    [junit] ------------- Standard Error -----------------
    [junit] NOTE: reproduce with: ant test -Dtestcase=TestLockFactory -Dtestmethod=testStressLocksNativeFSLockFactory
-Dtests.seed=9223296054268232625:-7758089421938554917
    [junit] NOTE: test params are: codec=RandomCodecProvider: {content=MockFixedIntBlock(blockSize=1397)},
locale=ar_MA, timezone=Indian/Antananarivo
    [junit] NOTE: all tests run in this JVM:
    [junit] [TestDateTools, Test2BTerms, TestAddIndexes, TestFilterIndexReader, TestIndexWriterExceptions,
TestIndexWriterMerging, TestMaxTermFrequency, TestParallelReaderEmptyIndex, TestParallelTermEnum,
TestPerSegmentDeletes, TestPersistentSnapshotDeletionPolicy, TestSegmentReader, TestStressAdvance,
TestConstantScoreQuery, TestDateFilter, TestDateSort, TestDocIdSet, TestNot, TestPrefixQuery,
TestSetNorm, TestTopScoreDocCollector, TestBasics, TestSpansAdvanced2, TestDirectory, TestLockFactory]
    [junit] NOTE: Linux 2.6.37-gentoo amd64/Sun Microsystems Inc. 1.6.0_25 (64-bit)/cpus=8,threads=1,free=136724544,total=292618240
{noformat}

that is caused by MockDirectoryWrapper behaving like Windows not deleting files if they are
still open. So there might be a segments_x file around but the _x.fnm has already been deleted.
That wasn't a problem before but since we now need FIs to decide if a segment is storing vectors
or not this file is required. 

To work around this I had to add some code to IndexFileDeleter which makes me worry a little.
Now I drop a commit-point if either I can't load the SIS or I can not load one of the FIs
from the loaded SI. I still try to delete all files of the "broken"?! segment though but the
question is if there could be cases where I should rather throw an exception in such a case.
Maybe some infoStream output would be helpful here to.

Any comments largely appreciated.

> Move hasVectors() & hasProx() responsibility out of SegmentInfo to FieldInfos 
> ------------------------------------------------------------------------------
>
>                 Key: LUCENE-2984
>                 URL: https://issues.apache.org/jira/browse/LUCENE-2984
>             Project: Lucene - Java
>          Issue Type: Improvement
>          Components: Index
>    Affects Versions: 4.0
>            Reporter: Simon Willnauer
>            Assignee: Simon Willnauer
>             Fix For: 4.0
>
>         Attachments: LUCENE-2984.patch, LUCENE-2984.patch, LUCENE-2984.patch
>
>
> Spin-off from LUCENE-2881 which had this change already but due to some random failures
related to this change I remove this part of the patch to make it more isolated and easier
to test. 

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org


Mime
View raw message