Mailing-List: contact solr-user-help@lucene.apache.org; run by ezmlm
Precedence: bulk
Reply-To: solr-user@lucene.apache.org
Received-SPF: pass (athena.apache.org: domain of yseeley@gmail.com designates
 209.85.146.177 as permitted sender)
DomainKey-Signature: a=rsa-sha1; c=nofws;
        d=gmail.com; s=gamma;
        h=mime-version:sender:reply-to:in-reply-to:references:date
         :x-google-sender-auth:message-id:subject:from:to:content-type
         :content-transfer-encoding;
        b=WgHzaju0w6A0Iyn+kr3R5CVFSm3YXajUaQAShY3TZ3GY4AnXZCKflaorVwa7wlw95A
         SI4hnRZrcuzcdhc5oICipUwavXFne27soarSmPRiZwyD1ZFf7BMxS50/BOPrb3RxgPHY
         UUsbuE+iz8aykFU3ZwJk02gzdn9JETjC19664=
MIME-Version: 1.0
Sender: yseeley@gmail.com
Reply-To: yonik@lucidimagination.com
In-Reply-To: <f516fcd80903031137p3a747b7fpfb03aa14bdd2b948@mail.gmail.com>
References: <f516fcd80903031137p3a747b7fpfb03aa14bdd2b948@mail.gmail.com>
Date: Tue, 3 Mar 2009 15:00:23 -0500
Message-ID: <c68e39170903031200i4b84b3eai6540221c1d1b17bb@mail.gmail.com>
Subject: Re: Very long commit time.
From: Yonik Seeley <yonik@lucidimagination.com>
To: solr-user@lucene.apache.org
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable

Looks like cache autowarming.
If you have statically defined warming queries in solrconfig.xml, you
could try setting autowarmCount=3D0 for all the caches.

-Yonik
http://www.lucidimagination.com


On Tue, Mar 3, 2009 at 2:37 PM, J=E9r=F4me Et=E9v=E9 <jerome.eteve@gmail.co=
m> wrote:
> Dear solr fans,
>
> =A0I have a solr index of roughly 8M docs and I have here a little
> problem when I commit some insertion into it.
>
> =A0The insert itself is very fast, but my commit takes 163 seconds.
>
> =A0Here's the solr trace the commit leaves:
>
> =A0INFO: start commit(optimize=3Dfalse,waitFlush=3Dfalse,waitSearcher=3Dt=
rue)
> =A003-Mar-2009 20:20:35 org.apache.solr.search.SolrIndexSearcher <init>
> INFO: Opening Searcher@7de212f9 main
> 03-Mar-2009 20:20:35 org.apache.solr.update.DirectUpdateHandler2 commit
> INFO: end_commit_flush
> 03-Mar-2009 20:20:35 org.apache.solr.search.SolrIndexSearcher warm
> INFO: autowarming Searcher@7de212f9 main from Searcher@732d8b11 main
> =A0 =A0 =A0 =A0filterCache{lookups=3D0,hits=3D0,hitratio=3D0.00,inserts=
=3D16,evictions=3D0,size=3D16,warmupTime=3D71641,cumulative_lookups=3D90,cu=
mulative_hits=3D68,cumulative_hitratio=3D0.75,cumulative_inserts=3D22,cumul=
ative_evictions=3D0}
> 03-Mar-2009 20:21:52 org.apache.solr.search.SolrIndexSearcher warm
> INFO: autowarming result for Searcher@7de212f9 main
> =A0 =A0 =A0 =A0filterCache{lookups=3D0,hits=3D0,hitratio=3D0.00,inserts=
=3D16,evictions=3D0,size=3D16,warmupTime=3D76905,cumulative_lookups=3D90,cu=
mulative_hits=3D68,cumulative_hitratio=3D0.75,cumulative_inserts=3D22,cumul=
ative_evictions=3D0}
> 03-Mar-2009 20:21:52 org.apache.solr.search.SolrIndexSearcher warm
> INFO: autowarming Searcher@7de212f9 main from Searcher@732d8b11 main
> =A0 =A0 =A0 =A0queryResultCache{lookups=3D24,hits=3D24,hitratio=3D1.00,in=
serts=3D32,evictions=3D0,size=3D32,warmupTime=3D82406,cumulative_lookups=3D=
6310,cumulative_hits=3D268,cumulative_hitratio=3D0.04,cumulative_inserts=3D=
6041,cumulative_evictions=3D5522}
> 03-Mar-2009 20:23:17 org.apache.solr.search.SolrIndexSearcher warm
> INFO: autowarming result for Searcher@7de212f9 main
> =A0 =A0 =A0 =A0queryResultCache{lookups=3D0,hits=3D0,hitratio=3D0.00,inse=
rts=3D32,evictions=3D0,size=3D32,warmupTime=3D85591,cumulative_lookups=3D63=
10,cumulative_hits=3D268,cumulative_hitratio=3D0.04,cumulative_inserts=3D60=
41,cumulative_evictions=3D5522}
> 03-Mar-2009 20:23:17 org.apache.solr.search.SolrIndexSearcher warm
> INFO: autowarming Searcher@7de212f9 main from Searcher@732d8b11 main
> =A0 =A0 =A0 =A0documentCache{lookups=3D720,hits=3D710,hitratio=3D0.98,ins=
erts=3D40,evictions=3D0,size=3D40,warmupTime=3D0,cumulative_lookups=3D41530=
8,cumulative_hits=3D283661,cumulative_hitratio=3D0.68,cumulative_inserts=3D=
131647,cumulative_evictions=3D131105}
> 03-Mar-2009 20:23:17 org.apache.solr.search.SolrIndexSearcher warm
> INFO: autowarming result for Searcher@7de212f9 main
> =A0 =A0 =A0 =A0documentCache{lookups=3D0,hits=3D0,hitratio=3D0.00,inserts=
=3D0,evictions=3D0,size=3D0,warmupTime=3D0,cumulative_lookups=3D415308,cumu=
lative_hits=3D283661,cumulative_hitratio=3D0.68,cumulative_inserts=3D131647=
,cumulative_evictions=3D131105}
> 03-Mar-2009 20:23:17 org.apache.solr.core.QuerySenderListener newSearcher
> INFO: QuerySenderListener sending requests to Searcher@7de212f9 main
>
> // Then the few warm up queries defined in solrconfig.xml
>
> INFO: Closing Searcher@732d8b11 main
> =A0 =A0 =A0 =A0filterCache{lookups=3D0,hits=3D0,hitratio=3D0.00,inserts=
=3D16,evictions=3D0,size=3D16,warmupTime=3D71641,cumulative_lookups=3D90,cu=
mulative_hits=3D68,cumulative_hitratio=3D0.75,cumulative_inserts=3D22,cumul=
ative_evictions=3D0}
> =A0 =A0 =A0 =A0queryResultCache{lookups=3D24,hits=3D24,hitratio=3D1.00,in=
serts=3D32,evictions=3D0,size=3D32,warmupTime=3D82406,cumulative_lookups=3D=
6310,cumulative_hits=3D268,cumulative_hitratio=3D0.04,cumulative_inserts=3D=
6041,cumulative_evictions=3D5522}
> =A0 =A0 =A0 =A0documentCache{lookups=3D720,hits=3D710,hitratio=3D0.98,ins=
erts=3D40,evictions=3D0,size=3D40,warmupTime=3D0,cumulative_lookups=3D41530=
8,cumulative_hits=3D283661,cumulative_hitratio=3D0.68,cumulative_inserts=3D=
131647,cumulative_evictions=3D131105}
> 03-Mar-2009 20:23:18 org.apache.solr.update.processor.LogUpdateProcessor =
finish
> INFO: {commit=3D} 0 163189
> 03-Mar-2009 20:23:18 org.apache.solr.core.SolrCore execute
> INFO: [jobs] webapp=3D/cjsolr path=3D/update/ params=3D{} status=3D0 QTim=
e=3D163189
>
>
> I'm sure I'm doing something wrong. Does this 163 seconds commit time
> have to do with the commit parameters :
> (optimize=3Dfalse,waitFlush=3Dfalse,waitSearcher=3Dtrue) =A0??
>
> Thanks for any help.
>
> Cheers !!
>
> Jerome.
>
> --
> Jerome Eteve.
>
> Chat with me live at http://www.eteve.net
>
> jerome@eteve.net
>