lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Sascha Janz" <Sascha.J...@gmx.net>
Subject Aw: RE: 回复: Never close IndexWriter/Reader?
Date Tue, 06 May 2014 08:35:45 GMT
Hello,
 
many thanks to your answers.  i modified our sources to leave the IndexWriter open. Also we
changed the commit strategy. may be we will do commit only at night. We used to create own
IndexReader for search. Now we changed this using SearcherManager which performs quite well.

 

We have an Index of nearly 50 GB. I am very curious how it will perform after the changes.
may it will be like unlocking a handbrake. i hope so.

 
Sascha

ps Uwe: i will be at your jax session next week.
 
 

Gesendet: Sonntag, 04. Mai 2014 um 10:47 Uhr
Von: "Uwe Schindler" <uwe@thetaphi.de>
An: java-user@lucene.apache.org
Betreff: RE: 回复: Never close IndexWriter/Reader?
Hi,

I would not do that. Reopening the NRT reader is not as expensive as committing (which also
fsyncs), but the current indexed data has to be written to a new segment on disk (called flush
in Lucene world). So reopenening periodically is the better approach (unless you have really
seldom accesses like one query per 1 minute). E.g., Elasticsearch reopens, by default, periodically
every second.

Uwe

-----
Uwe Schindler
H.-H.-Meier-Allee 63, D-28213 Bremen
http://www.thetaphi.de
eMail: uwe@thetaphi.de

> -----Original Message-----
> From: 308181687 [mailto:308181687@qq.com]
> Sent: Sunday, May 04, 2014 6:05 AM
> To: java-user
> Subject: 回复: Never close IndexWriter/Reader?
>
> Hi, Mike. Instead of periodically reopen NRT reader , I open/close it for every
> search query , will this result in performance issue?
>
>
> Thanks
> lubin
>
>
>
>
> ------------------ 原始邮件 ------------------
> 发件人: "Michael McCandless";<lucene@mikemccandless.com>;
> 发送时间: 2014年5月4日(星期天) 凌晨1:43
> 收件人: "Lucene Users"<java-user@lucene.apache.org>;
>
> 主题: Re: Never close IndexWriter/Reader?
>
>
>
> Just leave your IW open forever and periodically reopen your NRT
> reader. Be sure you close your old NRT reader after opening a new
> one; SearcherManager makes this easy when multiple threads are using
> the readers.
>
> Committing every 200 docs seems quite frequent: commit is very costly,
> and it's only needed for recovery (so you know which docs are in the
> index if the machine/OS crashes).
>
> Mike McCandless
>
> http://blog.mikemccandless.com[http://blog.mikemccandless.com]
>
>
> On Sat, May 3, 2014 at 11:46 AM, Sascha Janz <sascha.janz@gmx.net> wrote:
> > Hi,
> >
> >
> >
> > We use lucene 4.6, our application receives continuously new documents.
> > Mostly emails. We need the update near real time, so we open the
> IndexReader
> > with Directory.open and IndexWriter.
> >
> >
> >
> > Periodically we do a commit, e.g. every 200 documents.
> >
> >
> >
> > We used to close the IndexWriter on commit, and then open a new one. I
> read
> > in documentation this is quite expensive, tests also showed me this. So we
> > think about to only commit the IndexWriter and leave it open. So when
> should
> > we close the IndexWriter? I think about to leave it open and only close it,
> > when the server is shutting down. Is this a good idea? Any suggestions or
> > comments? Would be nice J
> >
> >
> >
> > Greetings
> >
> > Sascha
> >
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
>
> .


---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org
 

---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org


Mime
View raw message