lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Grant Ingersoll <>
Subject Re: FileNotFoundException in ConcurrentMergeScheduler
Date Wed, 11 Jun 2008 11:28:24 GMT

On Jun 11, 2008, at 6:00 AM, Michael McCandless wrote:

> Grant Ingersoll wrote:
>>> Is more than one thread adding documents to the index?
>> I don't believe so, but I am trying to reproduce.  I've only seen  
>> it once, and don't have a lot of details, other than I noticed it  
>> was on a specific file (.fdt) and was wondering if that was a  
>> factor or not.  That is, maybe Paul could reproduce it.
> I think your exception differs from Paul's in an important way.   
> Paul's exception means an entire segment (its CFS file) was deleted,  
> which is very easily caused by accidentally allowing 2 writers on  
> the index at once.  But in your case, SegmentReader successfully  
> opened the fnm file but then failed on the fdt, so, your segment  
> wasn't deleted (at least not entirely).  So I think something  
> different caused your exception.
>>> Any changes to the defaults in IndexWriter?
>> It's the SolrIndexWriter.
> OK.  But what does your solrconfig.xml look like?

It's pretty much the standard indexing setup that's on trunk.  Merge  
Factor 10, Max Buffered Docs 1000

>>> After seeing that exception, does also hit that  
>>> exception (ie, is/was the index corrupt)?  Or does it only happen  
>>> with BG merges?
>> Not sure, unfortunately, I don't have a lot of info yet.  The  
>> background exception happened during an optimize, if that matters  
>> at all
> OK.  It'd be very useful to know if index was really corrupt  
> (missing that segment) vs BG merge incorrectly, temporarily, thought  
> it was supposed to merge that segment.
> Is this a largish index?

couple million relatively small records

>  Like, would there be so many segments that optimize would be  
> running concurrent merges (> 2*mergeFactor segments)?  With  
> ConcurrentMergeScheduler, optimize is now able to run multiple  
> merges concurrently, if the index has enough segments.

I don't know.  Like I said, the only reason I posted was b/c I thought  
it sounded similar to Paul's.  I haven't been able to reproduce it yet.

> I'll run some stress tests, focusing on concurrency of merges during  
> optimize...
> Which OS & JRE are you using?

Linux, Java 1.6.0_05

To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message