lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From jm <>
Subject Re: trying to resolve error: after flush: fdx size mismatch
Date Thu, 15 Apr 2010 15:24:30 GMT
not sure if it matters, but I have to correct my statment, where this
has happened was both times win2008 R1 64bits, local filesystem.

I am trying to reproduce in my dev workstation but unable so far.

On Thu, Apr 15, 2010 at 10:11 AM, jm <> wrote:
> Hi Mike
> I have a server side, exposes a webservice with operations (this is
> simplified ignoring things non lucene related):
> - addToIndex(doc, date)
> - flushAllIndexes()
> addToIndex() adds the doc to the index, the index is chosen based on
> the date (simplified it's one index per day). To speed this up the
> webservice keeps a cache of IndexWriters.  If the index is already
> open it uses it, otherwise adds it to the cache (the index is created
> if it does not exist). I have a listener on the cache expiration so
> the index is closed when it is removed from the cache (the cache has
> only 10 items).
> flushAllIndexes() just gets all indexes open on the cache and closes them all.
> The webservice can serve many requests at the same time, but all
> operations dealing with lucene are synchonized. I guess this is most
> likely a bug in my synching code etc so I have verified it many times.
> Even trying to reproduce the issue by stopping threads in a debugger
> worked fine, synchronized methods were working as expected (I
> understand am probably  missing the case that triggers the issue).
> The last time this happened there was a single thread (in the client)
> calling addToIndex(doc, date) many times and then it called
> flushAllIndexes() The next second I got the error. The client
> developer assures me the call to flushAllIndexes() as in the same
> thread as addToIndex(doc, date) (anyway, it should work even if it was
> different threads of course).
> yes, local filesystem under Vista Business sp1 32b.
> Hope this helps. I am just now developing a client that pounds the
> server with some threads calling the methods all the time to try to
> reproduce the issue.
> javier
> On Thu, Apr 15, 2010 at 1:47 AM, Michael McCandless
> <> wrote:
>> Not good!
>> Can you describe how your threads work with Lucene?
>> Is this just a local filesystem (disk) under vista?
>> Mike
>> On Wed, Apr 14, 2010 at 7:41 AM, jm <> wrote:
>>> Hi,
>>> I am trying to chase an issue in our code and it is being quite
>>> difficult. We have seen two instances (see below) where we get the
>>> same error. I have been trying to reproduce but it has been impossible
>>> so far.I have several threads, some might be creating indices and
>>> adding documents, others closing indices etc.  In the list this
>>> message appear several times, in some cases it was a lucene bug (not
>>> my case, I dont have millions of docs here) and in other it seems bad
>>> multithreading. I have verified my multithreaded code for the 5th time
>>> and all seems well synchronized.
>>> So I am asking if somebody can give me any hint, maybe the fact that
>>> the first number is always a 1 means something??  I suspect the issue
>>> is when a new index is just created maybe.
>>> java.lang.RuntimeException: after flush: fdx size mismatch: 1 docs vs
>>> 103 length in bytes of _2.fdx file exists?=true
>>> org.apache.lucene.index.StoredFieldsWriter.closeDocStore(
>>> org.apache.lucene.index.DocFieldProcessor.closeDocStore(
>>> org.apache.lucene.index.DocumentsWriter.closeDocStore(
>>> org.apache.lucene.index.DocumentsWriter.flush(
>>> org.apache.lucene.index.IndexWriter.doFlushInternal(
>>> org.apache.lucene.index.IndexWriter.doFlush(
>>> org.apache.lucene.index.IndexWriter.flush(
>>> org.apache.lucene.index.IndexWriter.closeInternal(
>>> org.apache.lucene.index.IndexWriter.close(
>>> org.apache.lucene.index.IndexWriter.close(
>>> java.lang.RuntimeException: after flush: fdx size mismatch: 1 docs vs
>>> 8457 length in bytes of _0.fdx file exists?=true ...
>>> My env is vista, jdk1.6, lucene 2.9.1
>>> thanks in advance.
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail:
>>> For additional commands, e-mail:
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail:
>> For additional commands, e-mail:

To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message