lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Walter Underwood <wun...@wunderwood.org>
Subject Re: ArrayIndexOutOfBounds exception using FieldCache
Date Thu, 28 Oct 2010 14:59:52 GMT
How big is it? The Internet works pretty well for large files.

You can send a USB drive by snail mail.

wunder

On Oct 28, 2010, at 6:11 AM, <karl.wright@nokia.com> wrote:

> Talked with IT here - they don't recommend external transfers of this size.  So I think
we'd best try the "instrument and repeat" approach instead."
> 
> Karl
> 
> -----Original Message-----
> From: ext karl.wright@nokia.com [mailto:karl.wright@nokia.com] 
> Sent: Thursday, October 28, 2010 8:16 AM
> To: dev@lucene.apache.org
> Subject: RE: ArrayIndexOutOfBounds exception using FieldCache
> 
> It's on an internal Nokia machine, unfortunately, so the only way I can transfer it out
is with my credentials, or by email, which is definitely not going to work ;-).  But if you
can provide me with an account on a machine I'd be transferring it to, I may be able to scp
it from here.
> 
> Karl
> 
> 
> -----Original Message-----
> From: ext Michael McCandless [mailto:lucene@mikemccandless.com] 
> Sent: Thursday, October 28, 2010 7:50 AM
> To: dev@lucene.apache.org
> Subject: Re: ArrayIndexOutOfBounds exception using FieldCache
> 
> Fun fun :)
> 
> Is there anyway I can rsync/scp/ftp a copy of this index over....?
> 
> Failing that I can make some patches that we can iterate on...
> 
> Mike
> 
> On Thu, Oct 28, 2010 at 6:15 AM,  <karl.wright@nokia.com> wrote:
>> Not good indeed.
>> 
>> Synched to trunk, blew away old indexes, reindexed, same behavior.  So I think we've
got a problem, Houston. ;-)
>> 
>> Karl
>> 
>> -----Original Message-----
>> From: ext Michael McCandless [mailto:lucene@mikemccandless.com]
>> Sent: Wednesday, October 27, 2010 11:08 AM
>> To: dev@lucene.apache.org
>> Subject: Re: ArrayIndexOutOfBounds exception using FieldCache
>> 
>> Hmmm not good!
>> 
>> It could be you are hitting
>> https://issues.apache.org/jira/browse/LUCENE-2633?  That was fixed on
>> Sep 9, after your code.  Maybe try syncing up?
>> 
>> Mike
>> 
>> On Wed, Oct 27, 2010 at 9:21 AM,  <karl.wright@nokia.com> wrote:
>>> Hi Folks,
>>> 
>>> I just tried to index a data set that was probably 2x as large as the
>>> previous one I'd been using with the same code.  The indexing completed
>>> fine, although it was slower than I would have liked. ;-)  But the following
>>> problem occurs when I try to use FieldCache to look up an indexed and stored
>>> value:
>>> 
>>> java.lang.ArrayIndexOutOfBoundsException: -65406
>>>         at
>>> org.apache.lucene.util.PagedBytes$Reader.fillUsingLengthPrefix(PagedBytes.java:98)
>>>         at
>>> org.apache.lucene.search.FieldCacheImpl$DocTermsImpl.getTerm(FieldCacheImpl.java:918)
>>>         at ...
>>> 
>>> The code that does this has been working for quite some time and has been
>>> unmodified:
>>> 
>>>     /** Find a string field value, given the lucene ID, field name, and
>>> value.
>>>     */
>>>     protected String getStringValue(int luceneID, String fieldName)
>>>       throws IOException
>>>     {
>>>       // Find the right reader
>>>       final int idx = readerIndex(luceneID, starts, readers.length);
>>>       final int docBase = starts[idx];
>>>       final IndexReader reader = readers[idx];
>>> 
>>>       BytesRef ref =
>>> FieldCache.DEFAULT.getTerms(reader,fieldName).getTerm(luceneID-docBase,new
>>> BytesRef());
>>>       String rval = ref.utf8ToString();
>>>       //System.out.println(" Reading luceneID "+Integer.toString(luceneID)+"
>>> field "+fieldName+" with result '"+rval+"'");
>>>       return rval;
>>>     }
>>> 
>>>   }
>>> 
>>> I added a try/catch to see what values were going into the key line:
>>> 
>>> catch (RuntimeException e)
>>>     {
>>>         System.out.println("LuceneID = "+luceneID+",
>>> fieldName='"+fieldName+"', idx="+idx+", docBase="+docBase);
>>>         System.out.println("Readers = "+readers.length);
>>>         int i = 0;
>>>         while (i < readers.length)
>>>             {
>>>                 System.out.println(" Reader start "+i+" is "+starts[i]);
>>>                 i++;
>>>             }
>>>         throw e;
>>>     }
>>> 
>>> The resulting output was:
>>> 
>>> LuceneID = 34466856, fieldName='id', idx=0, docBase=0
>>> Readers = 1
>>>      Reader start 0 is 0
>>> 
>>> . which looks reasonable on the face of things.  This is a version of trunk
>>> from approximately 8/12/2010, so it is fairly old.  Was there a fix for a
>>> problem that could account for this behavior?  Should I simply synch up?  Or
>>> am I doing something wrong here?  The schema for the id field is:
>>> 
>>> <fieldType name="string_idx" class="solr.StrField" sortMissingLast="true"
>>> indexed="true" stored="true"/>
>>> <field name="id" type="string_idx" required="true"/>
>>> 
>>> Karl
>>> 
>> 
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
>> For additional commands, e-mail: dev-help@lucene.apache.org
>> 
>> 
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
>> For additional commands, e-mail: dev-help@lucene.apache.org
>> 
>> 
> 
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> For additional commands, e-mail: dev-help@lucene.apache.org
> 
> 
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> For additional commands, e-mail: dev-help@lucene.apache.org
> 
> 
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> For additional commands, e-mail: dev-help@lucene.apache.org
> 

--
Walter Underwood
Venture ASM, Troop 14, Palo Alto




---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org


Mime
View raw message