lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Michael McCandless <luc...@mikemccandless.com>
Subject Re: Are those runtime errors about the jdk, or lucene's jar, or my code?
Date Tue, 06 May 2008 18:54:21 GMT

Hi,

Could you run org.apache.lucene.index.CheckIndex on your index and  
post the result?

Are these exceptions easily reproduced starting from scratch (new  
index)?

More responses/questions below:

crspan wrote:
>
>
> -- OS: Linux lg99 2.6.5-7.276-smp #1 SMP Fri Sep 28 20:33:22 AKDT  
> 2007 x86_64 x86_64 x86_64 GNU/Linux
>
> -- Lucene:  2.3.2  (tried 2.2.0 as well, since the index was built  
> around 2.2.0, jdk1.6.0_01 )

Do you see these same exceptions when you run on Lucene 2.2.0?

> -- JDK:  Sun jdk1.6.0_06 ( from jdk-6u6-linux-x64.bin ) &  Sun  
> jdk1.5.0_15 ( from jdk-1_5_0_15-linux-amd64.bin)
> ( both installed locally in the user's home directory WITHOUT root  
> privilege.)
>
> -- Souce code:
>
> System.out.print("\n\n Range = "+range+"\nQuery = "+q.toString() 
> +"\n") ;
> tds = is.search( q, (Filter)null, range );
>
> -- Stack traces (1):
>
> Range = 500
> Query = TEXT:illeg^30.820824 TEXT:technolog^22.290413  
> TEXT:transfer^33.307804 TEXT:bipartisan^20.942562  
> TEXT:laboratori^18.500801 TEXT:norm^21.193087  
> TEXT:counterintellig^29.724474 TEXT:spi^19.285275  
> TEXT:lab^20.497044 TEXT:american^11.090684 TEXT:question^11.929131  
> TEXT:review^14.588552 TEXT:obtain^17.56319 TEXT:commun^12.5947275  
> TEXT:nation^10.737445 TEXT:offici^11.375352 TEXT:rep^17.646774  
> TEXT:contribut^15.35846 TEXT:report^11.633566  
> TEXT:congress^14.976282 TEXT:justic^16.433678 TEXT:govern^12.003913  
> TEXT:declassifi^31.553194 TEXT:campaign^14.959521  
> TEXT:inform^14.187338 TEXT:compani^13.717714  
> TEXT:classifi^23.613848 TEXT:washington^13.995003  
> TEXT:hugh^23.138725 TEXT:issu^14.177698 TEXT:space^18.239595 TEXT: 
> 1996^16.198292 TEXT:rocket^21.983511 TEXT:administr^17.11987  
> TEXT:satellit^21.777317 TEXT:nuclear^20.927034  
> TEXT:republican^18.929497 TEXT:committe^18.195517  
> TEXT:intellig^21.868582 TEXT:hous^17.309698 TEXT:democrat^18.528954  
> TEXT:investig^19.492653 TEXT:panel^22.208527 TEXT:senat^20.456139  
> TEXT:chines^19.726551 TEXT:sensit^23.52441 TEXT:secur^20.280426  
> TEXT:depart^21.874023 TEXT:missil^25.32581 TEXT:illeg^27.417799  
> TEXT:loral^41.9551 TEXT:transfer^33.933247
>
>
> QueryString:
> illeg^30.820824 technolog^22.290413 transfer^33.307804
> Error: java.lang.ArrayIndexOutOfBoundsException:  
> 132704java.lang.ArrayIndexOutOfBoundsException: 132704
>        at org.apache.lucene.search.BooleanScorer2 
> $Coordinator.coordFactor(BooleanScorer2.java:55)
>        at org.apache.lucene.search.BooleanScorer2.score 
> (BooleanScorer2.java:358)
>        at org.apache.lucene.search.BooleanScorer2.score 
> (BooleanScorer2.java:320)
>        at org.apache.lucene.search.IndexSearcher.search 
> (IndexSearcher.java:146)
>        at org.apache.lucene.search.IndexSearcher.search 
> (IndexSearcher.java:113)
>        at org.apache.lucene.search.Searcher.search(Searcher.java:132)
>        at org.cr.search.TrecQueryRelevanceFeedback.main 
> (TrecQueryRelevanceFeedback.java:785)

I'm not sure what could cause this one.

> -- Souce code:
>
>              TermFreqVector[] termsV = reader.getTermFreqVectors 
> (docID);
>
> -- Stack traces (2):
>
> Range = 500
> Query = TEXT:oceanograph^68.48028 TEXT:vessel^43.191563
>
>
> QueryString:
> oceanograph^68.48028 vessel^43.191563
> Error:  
> java.lang.ArrayIndexOutOfBoundsExceptionjava.lang.ArrayIndexOutOfBound 
> sException
>        at org.apache.lucene.index.TermVectorsReader.readTermVector 
> (TermVectorsReader.java:353)
>        at org.apache.lucene.index.TermVectorsReader.readTermVectors 
> (TermVectorsReader.java:287)
>        at org.apache.lucene.index.TermVectorsReader.get 
> (TermVectorsReader.java:232)
>        at org.apache.lucene.index.SegmentReader.getTermFreqVectors 
> (SegmentReader.java:981)
>        at org.cr.rf.RelevanceFeedback.RelFeedbackWeight 
> (RelevanceFeedback.java:145)
>        at org.cr.search.TrecQueryRelevanceFeedback.main 
> (TrecQueryRelevanceFeedback.java:789)
>

This looks like index corruption.  If you run CheckIndex it should  
detect this.  Did you hit any previous exceptions when writing to  
this index?  Is it possible to send me a copy of the index?

>
> --  Other Info:
>
> * This index can be searched in other programs in the same  
> environment.
> * The same program runs just fine in Window (1.6.0-b105 & 1.5.0_03- 
> b07), and a HP-UX (1.5.0.05) without those runtime errors.

This is very strange.  These same programs that create the above two  
exceptions, run fine on Windows & HP-UX?  Did you copy the index  
between these machines, or is it the very same index in a shared mount?

> -- My questions:
>
>
>
> ---------? What is your reading of those two stack traces?
>
>
> ---------? Where
>
> QueryString:
> illeg^30.820824 technolog^22.290413 transfer^33.307804
>
> got printed? It is NOT from my code, so is it part of Lucene's  
> error message? Which line in Lucene is for this print out?

I don't know!

>
> ---------?? As you can see, from the line
>
> System.out.print("\n\n Range = "+range+"\nQuery = "+q.toString() 
> +"\n") ;
>
> it just printed the 50 terms in the query:
>
> Range = 500
> Query = TEXT:illeg^30.820824 TEXT:technolog^22.290413  
> TEXT:transfer^33.307804 TEXT:bipartisan^20.942562  
> TEXT:laboratori^18.500801 TEXT:norm^21.193087  
> TEXT:counterintellig^29.724474 TEXT:spi^19.285275  
> TEXT:lab^20.497044 TEXT:american^11.090684 TEXT:question^11.929131  
> TEXT:review^14.588552 TEXT:obtain^17.56319 TEXT:commun^12.5947275  
> TEXT:nation^10.737445 TEXT:offici^11.375352 TEXT:rep^17.646774  
> TEXT:contribut^15.35846 TEXT:report^11.633566  
> TEXT:congress^14.976282 TEXT:justic^16.433678 TEXT:govern^12.003913  
> TEXT:declassifi^31.553194 TEXT:campaign^14.959521  
> TEXT:inform^14.187338 TEXT:compani^13.717714  
> TEXT:classifi^23.613848 TEXT:washington^13.995003  
> TEXT:hugh^23.138725 TEXT:issu^14.177698 TEXT:space^18.239595 TEXT: 
> 1996^16.198292 TEXT:rocket^21.983511 TEXT:administr^17.11987  
> TEXT:satellit^21.777317 TEXT:nuclear^20.927034  
> TEXT:republican^18.929497 TEXT:committe^18.195517  
> TEXT:intellig^21.868582 TEXT:hous^17.309698 TEXT:democrat^18.528954  
> TEXT:investig^19.492653 TEXT:panel^22.208527 TEXT:senat^20.456139  
> TEXT:chines^19.726551 TEXT:sensit^23.52441 TEXT:secur^20.280426  
> TEXT:depart^21.874023 TEXT:missil^25.32581 TEXT:illeg^27.417799  
> TEXT:loral^41.9551 TEXT:transfer^33.933247
>
> But how comes it became:
>
> QueryString:
> illeg^30.820824 technolog^22.290413 transfer^33.307804
>
>
> ---------? Overall, is this related to the JDK installation? wrong  
> bin file? without root? etc? Any advice to fix the problem?

Mike

---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org


Mime
View raw message