lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Michael McCandless <luc...@mikemccandless.com>
Subject Re: Problem of calling indexWriterConfig.clone()
Date Tue, 12 Aug 2014 14:04:09 GMT
IWC.clone is/was buggy ... just stop calling it and make a new IWC
from scratch each time in your application.

Mike McCandless

http://blog.mikemccandless.com


On Tue, Aug 12, 2014 at 8:37 AM, Sheng <shengcer@gmail.com> wrote:
> I think what you suggest probably will work, and I appreciate that. What I
> am a little concerned about is if Indexwriterconfig is completely stateless
> or not, meaning if i clone from the very original Indexwriterconfig, will I
> lose some info from the breakpoint? Maybe I don't need worry about it,
> since it is going to be removed in 4.9?
>
> On Tue, Aug 12, 2014 at 2:29 AM, Vitaly Funstein <vfunstein@gmail.com
> <javascript:_e(%7B%7D,'cvml','vfunstein@gmail.com');>> wrote:
>
>> I honestly don't understand what DWPT pool has to do with IndexWriterConfig
>> instances not being reusable for new IndexWriter instances. If you have the
>> need to open a new IndexWriter with the same configuration as the one you
>> used before, why not save the original config as the "template", then
>> simply do this for every IndexWriter instance you're creating:
>>
>> private final IndexWriterConfig masterCfg = new
>> IndexWriterConfig(Version.LUCENE_47, null);
>> // set whatever you need on this instance
>> .....
>>
>> IndexWriter writer = new IndexWriter(directory, masterCfg.clone());
>>
>> Wouldn't this just work? If not, could you paste the stack trace of the
>> exception you're getting?
>>
>>
>> On Mon, Aug 11, 2014 at 9:01 PM, Sheng <shengcer@gmail.com
>> <javascript:_e(%7B%7D,'cvml','shengcer@gmail.com');>> wrote:
>>
>> > From src code of DocumentsWriterPerThreadPool, the variable
>> > numThreadStatesActive seems to be always increasing, which explains why
>> > asserting on numThreadStatesActive == 0 before cloning this object
>> > fails. So what should be the most appropriate way of re-opening an
>> > indexwriter if what you have are the index directory plus the
>> > indexWriterConfig that the closed indexWriter has been using?
>> >
>> > BTW - I am reasonably sure calling indexWriterConfig.clone() in the
>> middle
>> > of indexing documents used to work for my code(same Lucene 4.7). It is
>> > since recently I had to do faceted indexing as well that this problem
>> > started to emerge. Is it related?
>> >
>> >
>> > On Mon, Aug 11, 2014 at 11:31 PM, Vitaly Funstein <vfunstein@gmail.com
>> <javascript:_e(%7B%7D,'cvml','vfunstein@gmail.com');>>
>> > wrote:
>> >
>> > > I only have the source to 4.6.1, but if you look at the constructor of
>> > > IndexWriter there, it looks like this:
>> > >
>> > >   public IndexWriter(Directory d, IndexWriterConfig conf) throws
>> > > IOException {
>> > >     conf.setIndexWriter(this); // prevent reuse by other instances
>> > >
>> > > The setter throws an exception if the configuration object has already
>> > been
>> > > used with another instance of IndexWriter. Therefore, it should be
>> cloned
>> > > before being used in the constructor of IndexWriter.
>> > >
>> > >
>> > > On Mon, Aug 11, 2014 at 7:12 PM, Sheng <shengcer@gmail.com
>> <javascript:_e(%7B%7D,'cvml','shengcer@gmail.com');>> wrote:
>> > >
>> > > > So the indexWriterConfig.clone() failed at this step:
>> > > > clone.indexerThreadPool = indexerThreadPool
>> > > > <
>> > > >
>> > >
>> >
>> http://grepcode.com/file/repo1.maven.org/maven2/org.apache.lucene/lucene-core/4.7.0/org/apache/lucene/index/LiveIndexWriterConfig.java#LiveIndexWriterConfig.0indexerThreadPool
>> > > > >
>> > > > .clone
>> > > > <
>> > > >
>> > >
>> >
>> http://grepcode.com/file/repo1.maven.org/maven2/org.apache.lucene/lucene-core/4.7.0/org/apache/lucene/index/DocumentsWriterPerThreadPool.java#DocumentsWriterPerThreadPool.clone%28%29
>> > > > >
>> > > > ();
>> > > >
>> > > > which then failed at this step in the "indexerThreadPool"
>> > > >
>> > > >
>> > > > if (numThreadStatesActive
>> > > > <
>> > > >
>> > >
>> >
>> http://grepcode.com/file/repo1.maven.org/maven2/org.apache.lucene/lucene-core/4.7.0/org/apache/lucene/index/DocumentsWriterPerThreadPool.java#DocumentsWriterPerThreadPool.0numThreadStatesActive
>> > > > >
>> > > > != 0) {
>> > > >
>> > > >     throw new IllegalStateException
>> > > > <
>> > > >
>> > >
>> >
>> http://grepcode.com/file/repository.grepcode.com/java/root/jdk/openjdk/6-b27/java/lang/IllegalStateException.java#IllegalStateException
>> > > > >("clone
>> > > > this object before it is used!");
>> > > >
>> > > > }
>> > > >
>> > > >
>> > > > There is a comment right above this:
>> > > > // We should only be cloned before being used:
>> > > >
>> > > > Does this mean whenever the indexWriter gets called for
>> > > > commit/prepareCommit, etc., the corresponding indexWriterConfig
>> object
>> > > > cannot be called with .clone() at all?
>> > > >
>> > > >
>> > > > On Mon, Aug 11, 2014 at 9:52 PM, Vitaly Funstein <
>> vfunstein@gmail.com <javascript:_e(%7B%7D,'cvml','vfunstein@gmail.com');>>
>> > > > wrote:
>> > > >
>> > > > > Looks like you have to clone it prior to using with any IndexWriter
>> > > > > instances.
>> > > > >
>> > > > >
>> > > > > On Mon, Aug 11, 2014 at 2:49 PM, Sheng <shengcer@gmail.com
>> <javascript:_e(%7B%7D,'cvml','shengcer@gmail.com');>> wrote:
>> > > > >
>> > > > > > I tried to create a clone of indexwriteconfig with
>> > > > > > "indexWriterConfig.clone()" for re-creating a new indexwriter,
>> but
>> > I
>> > > > > then I
>> > > > > > got this very annoying illegalstateexception: "clone this
object
>> > > before
>> > > > > it
>> > > > > > is used". Why does this exception happen, and how can I
get
>> around
>> > > it?
>> > > > > > Thanks!
>> > > > > >
>> > > > >
>> > > >
>> > >
>> >
>>

---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org


Mime
View raw message