lucene-solr-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Michael McCandless <luc...@mikemccandless.com>
Subject Re: background merge hit exception while optimizing index (SOLR 4.4.0)
Date Mon, 13 Jan 2014 20:15:35 GMT
I have trouble understanding J9's version strings ... but, is it
really from 2008?  You could be hitting a JVM bug; can you test
upgrading?

I don't have much experience with Solr faceting on optimized vs
unoptimized indices; maybe someone else can answer your question.

Lucene's facet module (not yet exposed through Solr) performance
shouldn't change much for optimized vs unoptimized indices.

Mike McCandless

http://blog.mikemccandless.com


On Mon, Jan 13, 2014 at 10:09 AM, Ralf Matulat
<ralf.matulat@bundestag.de> wrote:
>> java -version
> java version "1.6.0"
> Java(TM) SE Runtime Environment (build
> pxa6460sr3ifix-20090218_02(SR3+IZ43791+IZ43798))
> IBM J9 VM (build 2.4, J2RE 1.6.0 IBM J9 2.4 Linux amd64-64
> jvmxa6460-20081105_25433 (JIT enabled, AOT enabled)
> J9VM - 20081105_025433_LHdSMr
> JIT  - r9_20081031_1330
> GC   - 20081027_AB)
> JCL  - 20090218_01
>
> A question regarding to optimizing the index:
> As of SOLR 3.X we encountered massive performance improvements with facettet
> queries after optimizing an index. So we once started optimizing the indexes
> on a daily basis.
> With SOLR 4.X and the new index-format that is not true anymore?
>
> Btw: The checkIndex failed with 'java.io.FileNotFoundException:', I guess
> because I did not stopped the tomcat while checking. So SOLR created, merged
> and deleted some segments while checking. I will restart the check after
> stoppimg SOLR.
>
> Kind regards
> Ralf Matulat
>
>
>
>> Which version of Java are you using?
>>
>> That root cause exception is somewhat spooky: it's in the
>> ByteBufferIndexCode that handles an "UnderflowException", ie when a
>> small (maybe a few hundred bytes) read happens to span the 1 GB page
>> boundary, and specifically the exception happens on the final read
>> (curBuf.get(b, offset, len)).  Such page-spanning reads are very rare.
>>
>> The code looks fine to me though, and it's hard to explain how NPE (b
>> = null) could happen: that byte array is allocated in the
>> Lucene41PostingsReader.BlockDocsAndPositionsEnum class's ctor: encoded
>> = new byte[MAX_ENCODED_SIZE].
>>
>> Separately, you really should not have to optimize daily, if ever.
>>
>> Mike McCandless
>>
>> http://blog.mikemccandless.com
>>
>>
>>
>

Mime
View raw message