lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From <karl.wri...@nokia.com>
Subject RE: ArrayIndexOutOfBounds exception using FieldCache
Date Thu, 28 Oct 2010 13:11:01 GMT
Talked with IT here - they don't recommend external transfers of this size.  So I think we'd
best try the "instrument and repeat" approach instead."

Karl

-----Original Message-----
From: ext karl.wright@nokia.com [mailto:karl.wright@nokia.com] 
Sent: Thursday, October 28, 2010 8:16 AM
To: dev@lucene.apache.org
Subject: RE: ArrayIndexOutOfBounds exception using FieldCache

It's on an internal Nokia machine, unfortunately, so the only way I can transfer it out is
with my credentials, or by email, which is definitely not going to work ;-).  But if you can
provide me with an account on a machine I'd be transferring it to, I may be able to scp it
from here.

Karl
 

-----Original Message-----
From: ext Michael McCandless [mailto:lucene@mikemccandless.com] 
Sent: Thursday, October 28, 2010 7:50 AM
To: dev@lucene.apache.org
Subject: Re: ArrayIndexOutOfBounds exception using FieldCache

Fun fun :)

Is there anyway I can rsync/scp/ftp a copy of this index over....?

Failing that I can make some patches that we can iterate on...

Mike

On Thu, Oct 28, 2010 at 6:15 AM,  <karl.wright@nokia.com> wrote:
> Not good indeed.
>
> Synched to trunk, blew away old indexes, reindexed, same behavior.  So I think we've
got a problem, Houston. ;-)
>
> Karl
>
> -----Original Message-----
> From: ext Michael McCandless [mailto:lucene@mikemccandless.com]
> Sent: Wednesday, October 27, 2010 11:08 AM
> To: dev@lucene.apache.org
> Subject: Re: ArrayIndexOutOfBounds exception using FieldCache
>
> Hmmm not good!
>
> It could be you are hitting
> https://issues.apache.org/jira/browse/LUCENE-2633?  That was fixed on
> Sep 9, after your code.  Maybe try syncing up?
>
> Mike
>
> On Wed, Oct 27, 2010 at 9:21 AM,  <karl.wright@nokia.com> wrote:
>> Hi Folks,
>>
>> I just tried to index a data set that was probably 2x as large as the
>> previous one I'd been using with the same code.  The indexing completed
>> fine, although it was slower than I would have liked. ;-)  But the following
>> problem occurs when I try to use FieldCache to look up an indexed and stored
>> value:
>>
>> java.lang.ArrayIndexOutOfBoundsException: -65406
>>         at
>> org.apache.lucene.util.PagedBytes$Reader.fillUsingLengthPrefix(PagedBytes.java:98)
>>         at
>> org.apache.lucene.search.FieldCacheImpl$DocTermsImpl.getTerm(FieldCacheImpl.java:918)
>>         at ...
>>
>> The code that does this has been working for quite some time and has been
>> unmodified:
>>
>>     /** Find a string field value, given the lucene ID, field name, and
>> value.
>>     */
>>     protected String getStringValue(int luceneID, String fieldName)
>>       throws IOException
>>     {
>>       // Find the right reader
>>       final int idx = readerIndex(luceneID, starts, readers.length);
>>       final int docBase = starts[idx];
>>       final IndexReader reader = readers[idx];
>>
>>       BytesRef ref =
>> FieldCache.DEFAULT.getTerms(reader,fieldName).getTerm(luceneID-docBase,new
>> BytesRef());
>>       String rval = ref.utf8ToString();
>>       //System.out.println(" Reading luceneID "+Integer.toString(luceneID)+"
>> field "+fieldName+" with result '"+rval+"'");
>>       return rval;
>>     }
>>
>>   }
>>
>> I added a try/catch to see what values were going into the key line:
>>
>> catch (RuntimeException e)
>>     {
>>         System.out.println("LuceneID = "+luceneID+",
>> fieldName='"+fieldName+"', idx="+idx+", docBase="+docBase);
>>         System.out.println("Readers = "+readers.length);
>>         int i = 0;
>>         while (i < readers.length)
>>             {
>>                 System.out.println(" Reader start "+i+" is "+starts[i]);
>>                 i++;
>>             }
>>         throw e;
>>     }
>>
>> The resulting output was:
>>
>> LuceneID = 34466856, fieldName='id', idx=0, docBase=0
>> Readers = 1
>>      Reader start 0 is 0
>>
>> . which looks reasonable on the face of things.  This is a version of trunk
>> from approximately 8/12/2010, so it is fairly old.  Was there a fix for a
>> problem that could account for this behavior?  Should I simply synch up?  Or
>> am I doing something wrong here?  The schema for the id field is:
>>
>> <fieldType name="string_idx" class="solr.StrField" sortMissingLast="true"
>> indexed="true" stored="true"/>
>> <field name="id" type="string_idx" required="true"/>
>>
>> Karl
>>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> For additional commands, e-mail: dev-help@lucene.apache.org
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> For additional commands, e-mail: dev-help@lucene.apache.org
>
>

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org


Mime
View raw message