lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From jm <jmugur...@gmail.com>
Subject Re: trying to resolve error: after flush: fdx size mismatch
Date Thu, 15 Apr 2010 15:24:30 GMT
not sure if it matters, but I have to correct my statment, where this
has happened was both times win2008 R1 64bits, local filesystem.

I am trying to reproduce in my dev workstation but unable so far.

On Thu, Apr 15, 2010 at 10:11 AM, jm <jmuguruza@gmail.com> wrote:
> Hi Mike
>
> I have a server side, exposes a webservice with operations (this is
> simplified ignoring things non lucene related):
> - addToIndex(doc, date)
> - flushAllIndexes()
>
> addToIndex() adds the doc to the index, the index is chosen based on
> the date (simplified it's one index per day). To speed this up the
> webservice keeps a cache of IndexWriters.  If the index is already
> open it uses it, otherwise adds it to the cache (the index is created
> if it does not exist). I have a listener on the cache expiration so
> the index is closed when it is removed from the cache (the cache has
> only 10 items).
>
> flushAllIndexes() just gets all indexes open on the cache and closes them all.
>
> The webservice can serve many requests at the same time, but all
> operations dealing with lucene are synchonized. I guess this is most
> likely a bug in my synching code etc so I have verified it many times.
> Even trying to reproduce the issue by stopping threads in a debugger
> worked fine, synchronized methods were working as expected (I
> understand am probably  missing the case that triggers the issue).
>
> The last time this happened there was a single thread (in the client)
> calling addToIndex(doc, date) many times and then it called
> flushAllIndexes() The next second I got the error. The client
> developer assures me the call to flushAllIndexes() as in the same
> thread as addToIndex(doc, date) (anyway, it should work even if it was
> different threads of course).
>
> yes, local filesystem under Vista Business sp1 32b.
>
> Hope this helps. I am just now developing a client that pounds the
> server with some threads calling the methods all the time to try to
> reproduce the issue.
>
> javier
>
> On Thu, Apr 15, 2010 at 1:47 AM, Michael McCandless
> <lucene@mikemccandless.com> wrote:
>> Not good!
>>
>> Can you describe how your threads work with Lucene?
>>
>> Is this just a local filesystem (disk) under vista?
>>
>> Mike
>>
>> On Wed, Apr 14, 2010 at 7:41 AM, jm <jmuguruza@gmail.com> wrote:
>>> Hi,
>>>
>>> I am trying to chase an issue in our code and it is being quite
>>> difficult. We have seen two instances (see below) where we get the
>>> same error. I have been trying to reproduce but it has been impossible
>>> so far.I have several threads, some might be creating indices and
>>> adding documents, others closing indices etc.  In the list this
>>> message appear several times, in some cases it was a lucene bug (not
>>> my case, I dont have millions of docs here) and in other it seems bad
>>> multithreading. I have verified my multithreaded code for the 5th time
>>> and all seems well synchronized.
>>>
>>> So I am asking if somebody can give me any hint, maybe the fact that
>>> the first number is always a 1 means something??  I suspect the issue
>>> is when a new index is just created maybe.
>>>
>>> java.lang.RuntimeException: after flush: fdx size mismatch: 1 docs vs
>>> 103 length in bytes of _2.fdx file exists?=true
>>> org.apache.lucene.index.StoredFieldsWriter.closeDocStore(StoredFieldsWriter.java:97)
>>> org.apache.lucene.index.DocFieldProcessor.closeDocStore(DocFieldProcessor.java:50)
>>> org.apache.lucene.index.DocumentsWriter.closeDocStore(DocumentsWriter.java:380)
>>> org.apache.lucene.index.DocumentsWriter.flush(DocumentsWriter.java:581)
>>> org.apache.lucene.index.IndexWriter.doFlushInternal(IndexWriter.java:4294)
>>> org.apache.lucene.index.IndexWriter.doFlush(IndexWriter.java:4192)
>>> org.apache.lucene.index.IndexWriter.flush(IndexWriter.java:4183)
>>> org.apache.lucene.index.IndexWriter.closeInternal(IndexWriter.java:2190)
>>> org.apache.lucene.index.IndexWriter.close(IndexWriter.java:2153)
>>> org.apache.lucene.index.IndexWriter.close(IndexWriter.java:2117)
>>>
>>>
>>> java.lang.RuntimeException: after flush: fdx size mismatch: 1 docs vs
>>> 8457 length in bytes of _0.fdx file exists?=true ...
>>>
>>> My env is vista, jdk1.6, lucene 2.9.1
>>> thanks in advance.
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
>>> For additional commands, e-mail: java-user-help@lucene.apache.org
>>>
>>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
>> For additional commands, e-mail: java-user-help@lucene.apache.org
>>
>>
>

---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org


Mime
View raw message