Return-Path: Delivered-To: apmail-lucene-solr-user-archive@minotaur.apache.org Received: (qmail 6530 invoked from network); 3 Aug 2010 13:22:42 -0000 Received: from unknown (HELO mail.apache.org) (140.211.11.3) by 140.211.11.9 with SMTP; 3 Aug 2010 13:22:42 -0000 Received: (qmail 66614 invoked by uid 500); 3 Aug 2010 13:22:40 -0000 Delivered-To: apmail-lucene-solr-user-archive@lucene.apache.org Received: (qmail 66336 invoked by uid 500); 3 Aug 2010 13:22:36 -0000 Mailing-List: contact solr-user-help@lucene.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: solr-user@lucene.apache.org Delivered-To: mailing list solr-user@lucene.apache.org Received: (qmail 66322 invoked by uid 99); 3 Aug 2010 13:22:34 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 03 Aug 2010 13:22:34 +0000 X-ASF-Spam-Status: No, hits=0.0 required=10.0 tests=FREEMAIL_FROM,SPF_PASS,T_TO_NO_BRKTS_FREEMAIL X-Spam-Check-By: apache.org Received-SPF: pass (athena.apache.org: domain of paul.dlug@gmail.com designates 74.125.82.48 as permitted sender) Received: from [74.125.82.48] (HELO mail-ww0-f48.google.com) (74.125.82.48) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 03 Aug 2010 13:22:29 +0000 Received: by wwb39 with SMTP id 39so795240wwb.5 for ; Tue, 03 Aug 2010 06:22:08 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:received:in-reply-to :references:date:message-id:subject:from:to:content-type :content-transfer-encoding; bh=hnCsEXnupX3fGTRye9g2FqNKOR+FXg6Uh2hEDxGXSuQ=; b=HGzjF+FI0cEb03basV5UYoFqdSWkFKtTZw0K+2uQQJzl5S4+LafF+BPIF7NHoD2/Oo IWWnxioVRFemj9x+Wr8C0n/TqraplRs/ZAcTCJp2Z/xCOH2N8CSK8OqGYrVgYwEtq6iz AKF18RQ00FqCEzCODyX7WN0NLv3OGUqjiug34= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type:content-transfer-encoding; b=LNbsrPjg28gtakNaKwZpqtLChbR1h5PBXLOZZ/RALeCKDvEiNcQiTBPHA+0bDVr7Z9 dQD0+BTUQkATd3XWlgeo5szWa/4K1G5tiEiyBlaTeSQJjTsa+u24uNl3HHQtZu3zGAvz UZfogU2RZgLMhz1V0TAUcdppZBoGR9Jweotkg= MIME-Version: 1.0 Received: by 10.227.99.79 with SMTP id t15mr6262235wbn.209.1280841728296; Tue, 03 Aug 2010 06:22:08 -0700 (PDT) Received: by 10.216.15.17 with HTTP; Tue, 3 Aug 2010 06:22:08 -0700 (PDT) In-Reply-To: References: Date: Tue, 3 Aug 2010 09:22:08 -0400 Message-ID: Subject: Re: Queries with multiple wildcards failing in branch3x From: Paul Dlug To: solr-user@lucene.apache.org Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Sure, I'm reindexing now, I'll let you know how it goes. --Paul On Tue, Aug 3, 2010 at 9:05 AM, Michael McCandless wrote: > Ugh... I think there may still be a bug lurking. > > Karl is also still having problems, much further into his indexing proces= s. > > I'm hunting it now!! > > For the time being, I just disabled (committed to trunk & 3x) the > optimization that's causing the bug. =A0Can you update to 3x head (or > trunk head), remove your current index, and try again? > > Mike > > On Tue, Aug 3, 2010 at 8:52 AM, Paul Dlug wrote: >> Thanks, I updated to the latest version with the fix but I'm now >> getting another error when optimizing the index (or when searching >> certain fields). It mentions "unknown compression method" but I'm not >> using compressed fields at all. >> >> SEVERE: java.io.IOException: background merge hit exception: >> _a:C248670/19645 _l:C206701/14563 _m:C12186/100 _n:C11356 _o:C9945 >> _p:C9000 _q:C5704 _r:C2214 _s:C2000 _t:C1264 into _u [optimize] >> [mergeDocStores] >> =A0 =A0 =A0 =A0at org.apache.lucene.index.IndexWriter.optimize(IndexWrit= er.java:2392) >> =A0 =A0 =A0 =A0at org.apache.lucene.index.IndexWriter.optimize(IndexWrit= er.java:2320) >> =A0 =A0 =A0 =A0at org.apache.solr.update.DirectUpdateHandler2.commit(Dir= ectUpdateHandler2.java:403) >> =A0 =A0 =A0 =A0at org.apache.solr.update.processor.RunUpdateProcessor.pr= ocessCommit(RunUpdateProcessorFactory.java:85) >> =A0 =A0 =A0 =A0at org.apache.solr.handler.XMLLoader.processUpdate(XMLLoa= der.java:169) >> =A0 =A0 =A0 =A0at org.apache.solr.handler.XMLLoader.load(XMLLoader.java:= 69) >> =A0 =A0 =A0 =A0at org.apache.solr.handler.ContentStreamHandlerBase.handl= eRequestBody(ContentStreamHandlerBase.java:54) >> =A0 =A0 =A0 =A0at org.apache.solr.handler.RequestHandlerBase.handleReque= st(RequestHandlerBase.java:131) >> =A0 =A0 =A0 =A0at org.apache.solr.core.SolrCore.execute(SolrCore.java:13= 22) >> =A0 =A0 =A0 =A0at org.apache.solr.servlet.SolrDispatchFilter.execute(Sol= rDispatchFilter.java:341) >> =A0 =A0 =A0 =A0at org.apache.solr.servlet.SolrDispatchFilter.doFilter(So= lrDispatchFilter.java:244) >> =A0 =A0 =A0 =A0at org.apache.catalina.core.ApplicationFilterChain.intern= alDoFilter(ApplicationFilterChain.java:235) >> =A0 =A0 =A0 =A0at org.apache.catalina.core.ApplicationFilterChain.doFilt= er(ApplicationFilterChain.java:206) >> =A0 =A0 =A0 =A0at org.apache.catalina.core.StandardWrapperValve.invoke(S= tandardWrapperValve.java:233) >> =A0 =A0 =A0 =A0at org.apache.catalina.core.StandardContextValve.invoke(S= tandardContextValve.java:191) >> =A0 =A0 =A0 =A0at org.apache.catalina.core.StandardHostValve.invoke(Stan= dardHostValve.java:127) >> =A0 =A0 =A0 =A0at org.apache.catalina.valves.ErrorReportValve.invoke(Err= orReportValve.java:102) >> =A0 =A0 =A0 =A0at org.apache.catalina.core.StandardEngineValve.invoke(St= andardEngineValve.java:109) >> =A0 =A0 =A0 =A0at org.apache.catalina.connector.CoyoteAdapter.service(Co= yoteAdapter.java:298) >> =A0 =A0 =A0 =A0at org.apache.coyote.http11.Http11Processor.process(Http1= 1Processor.java:857) >> =A0 =A0 =A0 =A0at org.apache.coyote.http11.Http11Protocol$Http11Connecti= onHandler.process(Http11Protocol.java:588) >> =A0 =A0 =A0 =A0at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoE= ndpoint.java:489) >> =A0 =A0 =A0 =A0at java.lang.Thread.run(Thread.java:619) >> Caused by: org.apache.lucene.index.CorruptIndexException: field data >> are in wrong format: java.util.zip.DataFormatException: unknown >> compression method >> =A0 =A0 =A0 =A0at org.apache.lucene.index.FieldsReader.uncompress(Fields= Reader.java:585) >> =A0 =A0 =A0 =A0at org.apache.lucene.index.FieldsReader.addField(FieldsRe= ader.java:357) >> =A0 =A0 =A0 =A0at org.apache.lucene.index.FieldsReader.doc(FieldsReader.= java:239) >> =A0 =A0 =A0 =A0at org.apache.lucene.index.SegmentReader.document(Segment= Reader.java:894) >> =A0 =A0 =A0 =A0at org.apache.lucene.index.IndexReader.document(IndexRead= er.java:684) >> =A0 =A0 =A0 =A0at org.apache.lucene.index.SegmentMerger.copyFieldsWithDe= letions(SegmentMerger.java:410) >> =A0 =A0 =A0 =A0at org.apache.lucene.index.SegmentMerger.mergeFields(Segm= entMerger.java:338) >> =A0 =A0 =A0 =A0at org.apache.lucene.index.SegmentMerger.merge(SegmentMer= ger.java:159) >> =A0 =A0 =A0 =A0at org.apache.lucene.index.IndexWriter.mergeMiddle(IndexW= riter.java:4053) >> =A0 =A0 =A0 =A0at org.apache.lucene.index.IndexWriter.merge(IndexWriter.= java:3647) >> =A0 =A0 =A0 =A0at org.apache.lucene.index.ConcurrentMergeScheduler.doMer= ge(ConcurrentMergeScheduler.java:339) >> =A0 =A0 =A0 =A0at org.apache.lucene.index.ConcurrentMergeScheduler$Merge= Thread.run(ConcurrentMergeScheduler.java:407) >> Caused by: java.util.zip.DataFormatException: unknown compression method >> =A0 =A0 =A0 =A0at java.util.zip.Inflater.inflateBytes(Native Method) >> =A0 =A0 =A0 =A0at java.util.zip.Inflater.inflate(Inflater.java:238) >> =A0 =A0 =A0 =A0at java.util.zip.Inflater.inflate(Inflater.java:256) >> =A0 =A0 =A0 =A0at org.apache.lucene.document.CompressionTools.decompress= (CompressionTools.java:106) >> =A0 =A0 =A0 =A0at org.apache.lucene.index.FieldsReader.uncompress(Fields= Reader.java:582) >> =A0 =A0 =A0 =A0... 11 more >> >> >> >> On Mon, Aug 2, 2010 at 6:04 PM, Michael McCandless >> wrote: >>> This looks like the index corruption caused by a commit on Friday. >>> >>> See the thread I sent earlier with subject "heads up -- index >>> corruption on Solr/Lucene trunk/3.x branch". >>> >>> Mike >>> >>> On Mon, Aug 2, 2010 at 6:00 PM, Paul Dlug wrote: >>>> I'm running a recent build of branch3x (r981609), queries with >>>> multiple wildcards (e.g. a*b*c*) are failing with the exception below >>>> in the log. These queries worked fine for me with solr 1.4, known bug? >>>> >>>> >>>> SEVERE: java.lang.IndexOutOfBoundsException: Index: 114, Size: 39 >>>> =A0 =A0 =A0 =A0at java.util.ArrayList.RangeCheck(ArrayList.java:547) >>>> =A0 =A0 =A0 =A0at java.util.ArrayList.get(ArrayList.java:322) >>>> =A0 =A0 =A0 =A0at org.apache.lucene.index.FieldInfos.fieldInfo(FieldIn= fos.java:285) >>>> =A0 =A0 =A0 =A0at org.apache.lucene.index.FieldsReader.doc(FieldsReade= r.java:225) >>>> =A0 =A0 =A0 =A0at org.apache.lucene.index.SegmentReader.document(Segme= ntReader.java:894) >>>> =A0 =A0 =A0 =A0at org.apache.lucene.index.DirectoryReader.document(Dir= ectoryReader.java:527) >>>> =A0 =A0 =A0 =A0at org.apache.lucene.index.IndexReader.document(IndexRe= ader.java:684) >>>> =A0 =A0 =A0 =A0at org.apache.solr.search.SolrIndexReader.document(Solr= IndexReader.java:444) >>>> =A0 =A0 =A0 =A0at org.apache.solr.search.SolrIndexSearcher.doc(SolrInd= exSearcher.java:428) >>>> =A0 =A0 =A0 =A0at org.apache.solr.util.SolrPluginUtils.optimizePreFetc= hDocs(SolrPluginUtils.java:267) >>>> =A0 =A0 =A0 =A0at org.apache.solr.handler.component.QueryComponent.doP= refetch(QueryComponent.java:279) >>>> =A0 =A0 =A0 =A0at org.apache.solr.handler.component.QueryComponent.pro= cess(QueryComponent.java:186) >>>> =A0 =A0 =A0 =A0at org.apache.solr.handler.component.SearchHandler.hand= leRequestBody(SearchHandler.java:195) >>>> =A0 =A0 =A0 =A0at org.apache.solr.handler.RequestHandlerBase.handleReq= uest(RequestHandlerBase.java:131) >>>> =A0 =A0 =A0 =A0at org.apache.solr.core.SolrCore.execute(SolrCore.java:= 1322) >>>> =A0 =A0 =A0 =A0at org.apache.solr.servlet.SolrDispatchFilter.execute(S= olrDispatchFilter.java:341) >>>> =A0 =A0 =A0 =A0at org.apache.solr.servlet.SolrDispatchFilter.doFilter(= SolrDispatchFilter.java:244) >>>> =A0 =A0 =A0 =A0at org.apache.catalina.core.ApplicationFilterChain.inte= rnalDoFilter(ApplicationFilterChain.java:235) >>>> =A0 =A0 =A0 =A0at org.apache.catalina.core.ApplicationFilterChain.doFi= lter(ApplicationFilterChain.java:206) >>>> =A0 =A0 =A0 =A0at org.apache.catalina.core.StandardWrapperValve.invoke= (StandardWrapperValve.java:233) >>>> =A0 =A0 =A0 =A0at org.apache.catalina.core.StandardContextValve.invoke= (StandardContextValve.java:191) >>>> =A0 =A0 =A0 =A0at org.apache.catalina.core.StandardHostValve.invoke(St= andardHostValve.java:127) >>>> =A0 =A0 =A0 =A0at org.apache.catalina.valves.ErrorReportValve.invoke(E= rrorReportValve.java:102) >>>> =A0 =A0 =A0 =A0at org.apache.catalina.core.StandardEngineValve.invoke(= StandardEngineValve.java:109) >>>> =A0 =A0 =A0 =A0at org.apache.catalina.connector.CoyoteAdapter.service(= CoyoteAdapter.java:298) >>>> =A0 =A0 =A0 =A0at org.apache.coyote.http11.Http11Processor.process(Htt= p11Processor.java:857) >>>> =A0 =A0 =A0 =A0at org.apache.coyote.http11.Http11Protocol$Http11Connec= tionHandler.process(Http11Protocol.java:588) >>>> =A0 =A0 =A0 =A0at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JI= oEndpoint.java:489) >>>> =A0 =A0 =A0 =A0at java.lang.Thread.run(Thread.java:619) >>>> >>> >> >