lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Simon Willnauer <simon.willna...@googlemail.com>
Subject Re: ArrayIndexOutOfBounds exception using FieldCache
Date Thu, 28 Oct 2010 15:05:00 GMT
On Thu, Oct 28, 2010 at 4:59 PM, Walter Underwood <wunder@wunderwood.org> wrote:
> How big is it? The Internet works pretty well for large files.

Mike, pick the USB stick up during you next run :)

simon
>
> You can send a USB drive by snail mail.
>
> wunder
>
> On Oct 28, 2010, at 6:11 AM, <karl.wright@nokia.com> wrote:
>
>> Talked with IT here - they don't recommend external transfers of this size.  So
I think we'd best try the "instrument and repeat" approach instead."
>>
>> Karl
>>
>> -----Original Message-----
>> From: ext karl.wright@nokia.com [mailto:karl.wright@nokia.com]
>> Sent: Thursday, October 28, 2010 8:16 AM
>> To: dev@lucene.apache.org
>> Subject: RE: ArrayIndexOutOfBounds exception using FieldCache
>>
>> It's on an internal Nokia machine, unfortunately, so the only way I can transfer
it out is with my credentials, or by email, which is definitely not going to work ;-).  But
if you can provide me with an account on a machine I'd be transferring it to, I may be able
to scp it from here.
>>
>> Karl
>>
>>
>> -----Original Message-----
>> From: ext Michael McCandless [mailto:lucene@mikemccandless.com]
>> Sent: Thursday, October 28, 2010 7:50 AM
>> To: dev@lucene.apache.org
>> Subject: Re: ArrayIndexOutOfBounds exception using FieldCache
>>
>> Fun fun :)
>>
>> Is there anyway I can rsync/scp/ftp a copy of this index over....?
>>
>> Failing that I can make some patches that we can iterate on...
>>
>> Mike
>>
>> On Thu, Oct 28, 2010 at 6:15 AM,  <karl.wright@nokia.com> wrote:
>>> Not good indeed.
>>>
>>> Synched to trunk, blew away old indexes, reindexed, same behavior.  So I think
we've got a problem, Houston. ;-)
>>>
>>> Karl
>>>
>>> -----Original Message-----
>>> From: ext Michael McCandless [mailto:lucene@mikemccandless.com]
>>> Sent: Wednesday, October 27, 2010 11:08 AM
>>> To: dev@lucene.apache.org
>>> Subject: Re: ArrayIndexOutOfBounds exception using FieldCache
>>>
>>> Hmmm not good!
>>>
>>> It could be you are hitting
>>> https://issues.apache.org/jira/browse/LUCENE-2633?  That was fixed on
>>> Sep 9, after your code.  Maybe try syncing up?
>>>
>>> Mike
>>>
>>> On Wed, Oct 27, 2010 at 9:21 AM,  <karl.wright@nokia.com> wrote:
>>>> Hi Folks,
>>>>
>>>> I just tried to index a data set that was probably 2x as large as the
>>>> previous one I'd been using with the same code.  The indexing completed
>>>> fine, although it was slower than I would have liked. ;-)  But the following
>>>> problem occurs when I try to use FieldCache to look up an indexed and stored
>>>> value:
>>>>
>>>> java.lang.ArrayIndexOutOfBoundsException: -65406
>>>>         at
>>>> org.apache.lucene.util.PagedBytes$Reader.fillUsingLengthPrefix(PagedBytes.java:98)
>>>>         at
>>>> org.apache.lucene.search.FieldCacheImpl$DocTermsImpl.getTerm(FieldCacheImpl.java:918)
>>>>         at ...
>>>>
>>>> The code that does this has been working for quite some time and has been
>>>> unmodified:
>>>>
>>>>     /** Find a string field value, given the lucene ID, field name, and
>>>> value.
>>>>     */
>>>>     protected String getStringValue(int luceneID, String fieldName)
>>>>       throws IOException
>>>>     {
>>>>       // Find the right reader
>>>>       final int idx = readerIndex(luceneID, starts, readers.length);
>>>>       final int docBase = starts[idx];
>>>>       final IndexReader reader = readers[idx];
>>>>
>>>>       BytesRef ref =
>>>> FieldCache.DEFAULT.getTerms(reader,fieldName).getTerm(luceneID-docBase,new
>>>> BytesRef());
>>>>       String rval = ref.utf8ToString();
>>>>       //System.out.println(" Reading luceneID "+Integer.toString(luceneID)+"
>>>> field "+fieldName+" with result '"+rval+"'");
>>>>       return rval;
>>>>     }
>>>>
>>>>   }
>>>>
>>>> I added a try/catch to see what values were going into the key line:
>>>>
>>>> catch (RuntimeException e)
>>>>     {
>>>>         System.out.println("LuceneID = "+luceneID+",
>>>> fieldName='"+fieldName+"', idx="+idx+", docBase="+docBase);
>>>>         System.out.println("Readers = "+readers.length);
>>>>         int i = 0;
>>>>         while (i < readers.length)
>>>>             {
>>>>                 System.out.println(" Reader start "+i+" is "+starts[i]);
>>>>                 i++;
>>>>             }
>>>>         throw e;
>>>>     }
>>>>
>>>> The resulting output was:
>>>>
>>>> LuceneID = 34466856, fieldName='id', idx=0, docBase=0
>>>> Readers = 1
>>>>      Reader start 0 is 0
>>>>
>>>> . which looks reasonable on the face of things.  This is a version of trunk
>>>> from approximately 8/12/2010, so it is fairly old.  Was there a fix for
a
>>>> problem that could account for this behavior?  Should I simply synch up?
 Or
>>>> am I doing something wrong here?  The schema for the id field is:
>>>>
>>>> <fieldType name="string_idx" class="solr.StrField" sortMissingLast="true"
>>>> indexed="true" stored="true"/>
>>>> <field name="id" type="string_idx" required="true"/>
>>>>
>>>> Karl
>>>>
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
>>> For additional commands, e-mail: dev-help@lucene.apache.org
>>>
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
>>> For additional commands, e-mail: dev-help@lucene.apache.org
>>>
>>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
>> For additional commands, e-mail: dev-help@lucene.apache.org
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
>> For additional commands, e-mail: dev-help@lucene.apache.org
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
>> For additional commands, e-mail: dev-help@lucene.apache.org
>>
>
> --
> Walter Underwood
> Venture ASM, Troop 14, Palo Alto
>
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> For additional commands, e-mail: dev-help@lucene.apache.org
>
>

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org


Mime
View raw message