Mailing-List: contact dev-help@lucene.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@lucene.apache.org
Received-SPF: pass (athena.apache.org: domain of karl.wright@nokia.com
 designates 192.100.105.134 as permitted sender)
From: <karl.wright@nokia.com>
To: <dev@lucene.apache.org>
Subject: RE: ArrayIndexOutOfBounds exception using FieldCache
Thread-Topic: ArrayIndexOutOfBounds exception using FieldCache
Thread-Index: Act12eTm3OwTbKMeTYunrHY0v+60f////CkA//6d8QA=
Date: Thu, 28 Oct 2010 10:15:52 +0000
Message-ID: 
 <E137CC9DAAE88E489BA78565991902F303DC42@008-AM1MPN1-001.mgdnok.nokia.com>
References: 
 <E137CC9DAAE88E489BA78565991902F303D6DB@008-AM1MPN1-001.mgdnok.nokia.com>
 <AANLkTimu=Ooza=xfNOj-iF3NcnBP55F_-e69zFwC20so@mail.gmail.com>
In-Reply-To: <AANLkTimu=Ooza=xfNOj-iF3NcnBP55F_-e69zFwC20so@mail.gmail.com>
Accept-Language: en-US
Content-Language: en-US
Content-Type: text/plain; charset="iso-8859-1"
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0

Not good indeed.

Synched to trunk, blew away old indexes, reindexed, same behavior.  So I th=
ink we've got a problem, Houston. ;-)

Karl

-----Original Message-----
From: ext Michael McCandless [mailto:lucene@mikemccandless.com]=20
Sent: Wednesday, October 27, 2010 11:08 AM
To: dev@lucene.apache.org
Subject: Re: ArrayIndexOutOfBounds exception using FieldCache

Hmmm not good!

It could be you are hitting
https://issues.apache.org/jira/browse/LUCENE-2633?  That was fixed on
Sep 9, after your code.  Maybe try syncing up?

Mike

On Wed, Oct 27, 2010 at 9:21 AM,  <karl.wright@nokia.com> wrote:
> Hi Folks,
>
> I just tried to index a data set that was probably 2x as large as the
> previous one I'd been using with the same code.=A0 The indexing completed
> fine, although it was slower than I would have liked. ;-)=A0 But the foll=
owing
> problem occurs when I try to use FieldCache to look up an indexed and sto=
red
> value:
>
> java.lang.ArrayIndexOutOfBoundsException: -65406
> =A0=A0=A0=A0=A0=A0=A0 at
> org.apache.lucene.util.PagedBytes$Reader.fillUsingLengthPrefix(PagedBytes=
.java:98)
> =A0=A0=A0=A0=A0=A0=A0 at
> org.apache.lucene.search.FieldCacheImpl$DocTermsImpl.getTerm(FieldCacheIm=
pl.java:918)
> =A0=A0=A0=A0=A0=A0=A0 at ...
>
> The code that does this has been working for quite some time and has been
> unmodified:
>
> =A0=A0=A0 /** Find a string field value, given the lucene ID, field name,=
 and
> value.
> =A0=A0=A0 */
> =A0=A0=A0 protected String getStringValue(int luceneID, String fieldName)
> =A0=A0=A0=A0=A0 throws IOException
> =A0=A0=A0 {
> =A0=A0=A0=A0=A0 // Find the right reader
> =A0=A0=A0=A0=A0 final int idx =3D readerIndex(luceneID, starts, readers.l=
ength);
> =A0=A0=A0=A0=A0 final int docBase =3D starts[idx];
> =A0=A0=A0=A0=A0 final IndexReader reader =3D readers[idx];
>
> =A0=A0=A0=A0=A0 BytesRef ref =3D
> FieldCache.DEFAULT.getTerms(reader,fieldName).getTerm(luceneID-docBase,ne=
w
> BytesRef());
> =A0=A0=A0=A0=A0 String rval =3D ref.utf8ToString();
> =A0=A0=A0=A0=A0 //System.out.println(" Reading luceneID "+Integer.toStrin=
g(luceneID)+"
> field "+fieldName+" with result '"+rval+"'");
> =A0=A0=A0=A0=A0 return rval;
> =A0=A0=A0 }
>
> =A0 }
>
> I added a try/catch to see what values were going into the key line:
>
> catch (RuntimeException e)
> =A0=A0=A0 {
> =A0=A0=A0=A0=A0=A0=A0 System.out.println("LuceneID =3D "+luceneID+",
> fieldName=3D'"+fieldName+"', idx=3D"+idx+", docBase=3D"+docBase);
> =A0=A0=A0=A0=A0=A0=A0 System.out.println("Readers =3D "+readers.length);
> =A0=A0=A0=A0=A0=A0=A0 int i =3D 0;
> =A0=A0=A0=A0=A0=A0=A0 while (i < readers.length)
> =A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 {
> =A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 System.out.println(" Reader=
 start "+i+" is "+starts[i]);
> =A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 i++;
> =A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 }
> =A0=A0=A0=A0=A0=A0=A0 throw e;
> =A0=A0=A0 }
>
> The resulting output was:
>
> LuceneID =3D 34466856, fieldName=3D'id', idx=3D0, docBase=3D0
> Readers =3D 1
> =A0=A0=A0=A0 Reader start 0 is 0
>
> . which looks reasonable on the face of things.=A0 This is a version of t=
runk
> from approximately 8/12/2010, so it is fairly old.=A0 Was there a fix for=
 a
> problem that could account for this behavior?=A0 Should I simply synch up=
?=A0 Or
> am I doing something wrong here?=A0 The schema for the id field is:
>
> <fieldType name=3D"string_idx" class=3D"solr.StrField" sortMissingLast=3D=
"true"
> indexed=3D"true" stored=3D"true"/>
> <field name=3D"id" type=3D"string_idx" required=3D"true"/>
>
> Karl
>

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org