lucene-solr-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ben Shlomo, Yatir" <>
Subject *Very* slow Commit after upgrading to solr 1.3
Date Mon, 29 Sep 2008 09:33:45 GMT


I am running on widows 64 bit ...
I have upgraded to solr 1.3 in order to use the distributed search.

I haven't changed the solrConfig and the schema xml files during the

I am indexing ~ 350K documents (each one is about 0.5 KB in size)

The indexing takes a reasonable amount of time (350 seconds)

See tomcat log:

INFO: {add=[8x-wbTscWftuu1sVWpdnGw==, VOu1eSv0obBl1xkj2jGjIA==,
YkOm-nKPrTVVVyeCZM4-4A==, rvaq_TyYsqt3aBc0KKDVbQ==,
9NdzWXsErbF_5btyT1JUjw==, ...(398728 more)]} 0 349875


But when I commit it takes more than an hour ! (5000 seconds!, the
optimize after the commit took 14 seconds)

INFO: start commit(optimize=false,waitFlush=false,waitSearcher=true)


p.s. its not a machine problem I moved to another machine and the same
thing happened

I noticed something very strange during the time I wait for the commit:

While the solr index is 210MB in size

In the windows task manager I noticed that the java process is making a
HUGE amounts of IO reads:

It reads more than 350 GB ! (- which takes a lot of time.)

The process is constantly taking 25% of the cpu resources.

All my autowarmCount in Solrconfig  file do not exceed 256...


Any more ideas to check?





Here is part of my solrConfig file:

- <file:///C:\dss1\SolrHome\conf\solrconfig.xml##>  < - <indexDefaults>

- <!--  Values here affect all index writers and act as a default unless










- <mainIndex>

- <!--  options specific to the main on-disk lucene index 







- <!--  If true, unlock any held write or commit locks on startup. 

         This defeats the locking mechanism that allows multiple

         processes to safely access a lucene index, and should be

         used with care. 









Yatir Ben-shlomo | eBay, Inc. | Classification Track,
(Israel) | w: +972-9-892-1373 |  email: |


  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message