lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From andi rexha <a_re...@hotmail.com>
Subject RE: Multi-thread indexing, should the commit be called from each thread?
Date Thu, 22 May 2014 08:35:44 GMT
I was certainly asking about lucene!

> From: jack@basetechnology.com
> To: java-user@lucene.apache.org
> Subject: Re: Multi-thread indexing, should the commit be called from each thread?
> Date: Wed, 21 May 2014 11:53:46 -0400
> 
> (Was this supposed to be a java-user/Lucene question or a Solr question?!)
> 
> -- Jack Krupansky
> 
> -----Original Message----- 
> From: Erick Erickson
> Sent: Wednesday, May 21, 2014 10:58 AM
> To: java-user
> Subject: Re: Multi-thread indexing, should the commit be called from each 
> thread?
> 
> I'll be more emphatic than Shai; you should _definitely_ not commit
> from each thread, especially if you are doing a hard commit with
> openSearcher=true or a soft commit.
> 
> In either case you open a new searcher which fires all your
> autowarming queries which.. IOW they're expensive operations.
> 
> More than you ever wanted to know about hard and soft commits is here:
> http://searchhub.org/2013/08/23/understanding-transaction-logs-softcommit-and-commit-in-sorlcloud/
> 
> FWIW,
> Erick
> 
> On Wed, May 21, 2014 at 6:12 AM, Shai Erera <serera@gmail.com> wrote:
> > You don't need to commit from each thread, you can definitely commit when
> > all threads are done. In general, you should commit only when you want to
> > ensure the data is "safe" on disk.
> >
> > Shai
> >
> >
> > On Wed, May 21, 2014 at 2:58 PM, andi rexha <a_rexha@hotmail.com> wrote:
> >
> >> Hi!
> >> I have a question about multi-thread indexing. When I perform a
> >> Multi-thread indexing, should I commit from each thread that I add
> >> documents or the commit should be done only when all the threads are done
> >> with their indexing task?
> >>
> >> Thank you!
> >>
> 
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org 
> 
> 
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
> 
 		 	   		  
Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message