lucene-solr-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From sunnyfr <johanna...@gmail.com>
Subject Re: Problem for replication : segment optimized automaticly
Date Mon, 23 Mar 2009 14:48:47 GMT

Where do you see that I can maybe check in my code, is it in updateHandler2 ?

thanks,


Shalin Shekhar Mangar wrote:
> 
> Lucene will automatically merge segments when they exceed the mergeFactor.
> This may be one reason but I'm not sure.
> 
> I checked DataImportHandler's code again. It won't optimize if
> optimize=false is specified.
> 
> On Mon, Mar 23, 2009 at 12:43 AM, sunnyfr <johanna.34@gmail.com> wrote:
> 
>>
>> Do you have any idea ???
>> :(
>>
>> cheer,
>>
>>
>> sunnyfr wrote:
>> >
>> > Hi everybody ... still me :)
>> > hoo happy day :)
>> >
>> > Just, I dont get where I miss something, I will try to be clear.
>> >
>> > this is my index folder (and we can notice the evolution according to
>> the
>> > delta import every 30mn) :
>> >
>> > root@search-01:/data/solr# ls video/data/index/
>> > _2bel.fdt  _2bel.fnm  _2bel.nrm  _2bel.tii  _2bel.tvd  _2bel.tvx
>> > _2bem.fdt  _2bem.fnm  _2bem.nrm  _2bem.tii  _2bem.tvd  _2bem.tvx
>> > _2ben.frq  _2ben.prx  _2ben.tis  _2beo.fdx  segments.gen
>> > _2bel.fdx  _2bel.frq  _2bel.prx  _2bel.tis  _2bel.tvf  _2bel_1.del
>> > _2bem.fdx  _2bem.frq  _2bem.prx  _2bem.tis  _2bem.tvf  _2ben.fnm
>> > _2ben.nrm  _2ben.tii  _2beo.fdt  _2beo.fnm  segments_230x
>> > root@search-01:/data/solr# ls video/data/index/
>> > _2bel.fdt  _2bel.frq  _2bel.tii  _2bel.tvf    _2bem.fdt  _2bem.frq
>> > _2bem.tii  _2bem.tvf  _2ben.frq  _2ben.tii  _2beo.fdx  _2beo.nrm
>> > _2beo.tis  _2beo.tvx
>> > _2bel.fdx  _2bel.nrm  _2bel.tis  _2bel.tvx    _2bem.fdx  _2bem.nrm
>> > _2bem.tis  _2bem.tvx  _2ben.nrm  _2ben.tis  _2beo.fnm  _2beo.prx
>> > _2beo.tvd  segments.gen
>> > _2bel.fnm  _2bel.prx  _2bel.tvd  _2bel_1.del  _2bem.fnm  _2bem.prx
>> > _2bem.tvd  _2ben.fnm  _2ben.prx  _2beo.fdt  _2beo.frq  _2beo.tii
>> > _2beo.tvf  segments_230x
>> > root@search-01:/data/solr# ls video/data/index/
>> > _2beo.fdt  _2beo.fdx  _2beo.fnm  _2beo.frq  _2beo.nrm  _2beo.prx
>> > _2beo.tii  _2beo.tis  _2beo.tvd  _2beo.tvf  _2beo.tvx  segments.gen
>> > segments_230y
>> >
>> > So as you can notice my segments increased which is perfect for my
>> > replication (faster to get JUST last segments)
>> > But like you can see in my last ls my segment has been optimized.
>> >
>> > Like I can notice in my log :
>> > Mar 19 15:42:37 search-01 jsvc.exec[23255]: Mar 19, 2009 3:42:37 PM
>> > org.apache.solr.handler.dataimport.DocBuilder commit INFO: Full Import
>> > completed successfully without optimization
>> > Mar 19 15:42:37 search-01 jsvc.exec[23255]: Mar 19, 2009 3:42:37 PM
>> > org.apache.solr.update.DirectUpdateHandler2 commit INFO: start
>> > commit(optimize=true,waitFlush=false,waitSearcher=true)
>> >
>> > But I didn't fire any optimize, my delta-import is fired like :
>> > /solr/video/dataimport?command=delta-import&optimize=false
>> >
>> > Solrconfig.xml : autocommit turnd off
>> > <!--<luceneAutoCommit>false</luceneAutoCommit>-->
>> > <!-- <commitLockTimeout>18500</commitLockTimeout>-->
>> > <!-- autocommit pending docs if certain criteria are met
>> >   <autoCommit>
>> >   <maxDocs>10000</maxDocs>
>> >   </autoCommit>
>> > -->
>> >
>> > Maybe it comes from lucene parameters?
>> >     <!-- options specific to the main on-disk lucene index -->
>> >     <useCompoundFile>false</useCompoundFile>
>> >     <ramBufferSizeMB>50</ramBufferSizeMB>
>> >     <mergeFactor>50</mergeFactor>
>> >     <!-- Deprecated -->
>> >     <!--<maxBufferedDocs>1000</maxBufferedDocs>-->
>> >     <maxMergeDocs>2147483647</maxMergeDocs>
>> >     <maxFieldLength>10000</maxFieldLength>
>> >
>> > Thanks a lot for your help,
>> > Sunny
>> >
>> >
>> >
>> >
>>
>> --
>> View this message in context:
>> http://www.nabble.com/Problem-for-replication-%3A-segment-optimized-automaticly-tp22601442p22649412.html
>> Sent from the Solr - User mailing list archive at Nabble.com.
>>
>>
> 
> 
> -- 
> Regards,
> Shalin Shekhar Mangar.
> 
> 

-- 
View this message in context: http://www.nabble.com/Problem-for-replication-%3A-segment-optimized-automaticly-tp22601442p22661653.html
Sent from the Solr - User mailing list archive at Nabble.com.


Mime
View raw message