lucene-solr-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Lance Norskog <goks...@gmail.com>
Subject Re: Solr Index Lock Issue
Date Sat, 17 Apr 2010 21:20:44 GMT
The commit call can return before Solr is completely finished with
everything it does after a commit. This tail processing can build up
with successive commits until Solr is basically stuck.

Commit should use waitFlush=true. This makes the commit call wait
until the disk manipulation is done. If you add 'waitSearch=true' and
'expungeDeletes=true', this will wait completely until Solr is ready
to do more indexing.

"Solr just hangs" is not enough to go on. Solr/Lucene have processing
where it takes increasing amounts of time to index data, Is the CPU
busy? Is the disk busy? Does 'jhat' or 'visualgc' show memory
activity?

On 4/16/10, Otis Gospodnetic <otis_gospodnetic@yahoo.com> wrote:
> Hi,
>
> What you are doing sounds fine.  You don't need to commit while indexing,
> though, just commit/optimize at the end.  I'm not saying this will solve
> your problem, but give it a try.
>
> Otis
> ----
> Sematext :: http://sematext.com/ :: Solr - Lucene - Nutch
> Hadoop ecosystem search :: http://search-hadoop.com/
>
>
>
> ----- Original Message ----
>> From: "Sethi, Parampreet" <parampreet.sethi@corp.aol.com>
>> To: solr-user@lucene.apache.org
>> Sent: Fri, April 16, 2010 1:13:57 PM
>> Subject: Solr Index Lock Issue
>>
>> Hi All,
>
> We are facing the issue with the Solr server in the DMOZ data
>> migration.
> The Solr has 0 records when the migration starts and the data is
>> added
> into Solr in the batches of 20000 records. The commit is called on
>> Solr
> after 20k records are processed.
>
> While commiting the data into
>> Solr, a lucene lock file is created in the
> <Solr_Home>/data/index
>> folder which is automatically released once the
> successful commit happens.
>> But after 4-5 batches, the lock file remains
> there and Solr just hangs and
>> does not add any new records. Some times
> the whole migration goes through
>> without any errors.
>
> Kindly let me know in case some setting needs to be
>> required on Solr
> side, which ensures that until the Solr commits the index,
>> the next set
> of records should not be added.
>
> Thanks,
> Param
>


-- 
Lance Norskog
goksron@gmail.com

Mime
View raw message