From java-user-return-34385-apmail-lucene-java-user-archive=lucene.apache.org@lucene.apache.org Wed Jun 11 10:01:25 2008 Return-Path: Delivered-To: apmail-lucene-java-user-archive@www.apache.org Received: (qmail 48166 invoked from network); 11 Jun 2008 10:01:24 -0000 Received: from hermes.apache.org (HELO mail.apache.org) (140.211.11.2) by minotaur.apache.org with SMTP; 11 Jun 2008 10:01:24 -0000 Received: (qmail 95542 invoked by uid 500); 11 Jun 2008 10:01:19 -0000 Delivered-To: apmail-lucene-java-user-archive@lucene.apache.org Received: (qmail 95516 invoked by uid 500); 11 Jun 2008 10:01:19 -0000 Mailing-List: contact java-user-help@lucene.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: java-user@lucene.apache.org Delivered-To: mailing list java-user@lucene.apache.org Received: (qmail 95505 invoked by uid 99); 11 Jun 2008 10:01:19 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 11 Jun 2008 03:01:19 -0700 X-ASF-Spam-Status: No, hits=1.2 required=10.0 tests=SPF_NEUTRAL X-Spam-Check-By: apache.org Received-SPF: neutral (athena.apache.org: local policy) Received: from [72.14.204.236] (HELO qb-out-0506.google.com) (72.14.204.236) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 11 Jun 2008 10:00:28 +0000 Received: by qb-out-0506.google.com with SMTP id a16so1106976qbd.23 for ; Wed, 11 Jun 2008 03:00:44 -0700 (PDT) Received: by 10.114.36.4 with SMTP id j4mr6186472waj.156.1213178444291; Wed, 11 Jun 2008 03:00:44 -0700 (PDT) Received: from ?10.17.4.4? ( [72.93.214.93]) by mx.google.com with ESMTPS id 6sm2877846ywn.0.2008.06.11.03.00.43 (version=TLSv1/SSLv3 cipher=RC4-MD5); Wed, 11 Jun 2008 03:00:43 -0700 (PDT) Mime-Version: 1.0 (Apple Message framework v753) In-Reply-To: <6E16D593-87FE-43A4-AC5F-5E1BED594B2B@apache.org> References: <2CF563C9-66FC-424E-9069-AFDC7307A856@apache.org> <3554C441-6C62-4E34-BDB9-0606D99EEA6F@mikemccandless.com> <6E16D593-87FE-43A4-AC5F-5E1BED594B2B@apache.org> Content-Type: text/plain; charset=US-ASCII; delsp=yes; format=flowed Message-Id: <00C66825-EB8B-4B08-82B1-72CF53E25C33@mikemccandless.com> Content-Transfer-Encoding: 7bit From: Michael McCandless Subject: Re: FileNotFoundException in ConcurrentMergeScheduler Date: Wed, 11 Jun 2008 06:00:48 -0400 To: java-user@lucene.apache.org X-Mailer: Apple Mail (2.753) X-Virus-Checked: Checked by ClamAV on apache.org Grant Ingersoll wrote: >> Is more than one thread adding documents to the index? > > I don't believe so, but I am trying to reproduce. I've only seen > it once, and don't have a lot of details, other than I noticed it > was on a specific file (.fdt) and was wondering if that was a > factor or not. That is, maybe Paul could reproduce it. I think your exception differs from Paul's in an important way. Paul's exception means an entire segment (its CFS file) was deleted, which is very easily caused by accidentally allowing 2 writers on the index at once. But in your case, SegmentReader successfully opened the fnm file but then failed on the fdt, so, your segment wasn't deleted (at least not entirely). So I think something different caused your exception. >> Any changes to the defaults in IndexWriter? > > It's the SolrIndexWriter. OK. But what does your solrconfig.xml look like? >> >> >> After seeing that exception, does IndexReader.open also hit that >> exception (ie, is/was the index corrupt)? Or does it only happen >> with BG merges? > > Not sure, unfortunately, I don't have a lot of info yet. The > background exception happened during an optimize, if that matters > at all OK. It'd be very useful to know if index was really corrupt (missing that segment) vs BG merge incorrectly, temporarily, thought it was supposed to merge that segment. Is this a largish index? Like, would there be so many segments that optimize would be running concurrent merges (> 2*mergeFactor segments)? With ConcurrentMergeScheduler, optimize is now able to run multiple merges concurrently, if the index has enough segments. I'll run some stress tests, focusing on concurrency of merges during optimize... Which OS & JRE are you using? Mike --------------------------------------------------------------------- To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org For additional commands, e-mail: java-user-help@lucene.apache.org