lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Dinh <pcd...@gmail.com>
Subject Re: deleteDocuments() does not work
Date Tue, 03 Nov 2009 10:21:24 GMT
Hi Michael,

> Does that mean you no longer see the original problem (changes not
> being reflected)?

Yes. The deleted documents do not appear in search results any more. I am
not sure that if they are flushed to disk
at that time yet but at least there is a sign that they are "deleted". I
have stopped and started the servlet engine to ensure
that deleted document is no longer there. I think that Lucene requires the
previously opened IndexReader be closed before changes
can be reflected.

> You get the entrySet from the Map, you then iterate over its
> Map.Entry, then you replace in your original map some entries (the
> ones that are opened).  So, you are modifying a Java collection while
> iterating over elements from its Set view... I just don't know if
> that's safe (anyone?)

I am a bit skeptical about my approach. Because the IndexSearchers can be
used by other threads (requests) at the same time.
So when I close them, some users can be affected. I will find a better way
to do it.

Also, because reload() is synchronized so there is a single thread accessing
it only. So I think that there will be no ConcurrentModificationException

> Would be good to instrument/debug and confirm
> that the precise IndexReader that's searching the Directory your
> IndexWriter just committed to, is in fact reopened.

Do you think that these code are enough

IndexReader oldReader = searcher.getIndexReader();
IndexReader newReader = oldReader.reopen(true);

if (newReader != oldReader) {
    oldReader.close();
    searcher.close();
    searchers.put(entry.getKey(), new IndexSearcher(newReader));
}

Thanks,

Dinh

On Tue, Nov 3, 2009 at 4:50 PM, Michael McCandless <
lucene@mikemccandless.com> wrote:

> On Tue, Nov 3, 2009 at 4:24 AM, Dinh <pcdinh@gmail.com> wrote:
> > Hi Michael,
> >
> > Thank a lot for your advice
> >
> >> Can you verify you are in fact reopening the reader that's reading the
> >> same Directory the writer is writing to?
> >
> > Yes. I have a single and configurable index path. So I can not make a
> > mistake here
>
> OK.
>
> >> Also, you are failing to close the old reader after opening a new one.
> >> This shouldn't cause the issue you're seeing, but, will lead
> >> eventually to OOME or file descriptor exhaustion.
> >
> > I have rewritten the method as follows
> >
> >    /**
> >     * Reloads searchers after index is changed (added, deleted or
> updated).
> >     */
> >    public static synchronized void reload() {
> >
> >        Set<Map.Entry<String, IndexSearcher>> set = searchers.entrySet();
> >
> >        for (Map.Entry<String, IndexSearcher> entry : set) {
> >            try {
> >                IndexSearcher searcher = entry.getValue();
> >                IndexReader oldReader = searcher.getIndexReader();
> >                IndexReader newReader = oldReader.reopen(true);
> >
> >                if (newReader != oldReader) {
> >                    oldReader.close();
> >                    searcher.close();
> >                    searchers.put(entry.getKey(), new
> > IndexSearcher(newReader));
> >                }
> >            } catch (Exception e) {
> >                log.warn(e.getMessage(), e);
> >            }
> >        }
> >    }
>
> Your reload method looks better now!  (You are now closing the old reader).
>
> > And it works now.
>
> Does that mean you no longer see the original problem (changes not
> being reflected)?
>
> >> Finally, are you sure the iteration over the Map entries, that
> >> overwrites each entry, is safe?
> >
> > Do you think that my iteration is safe now? At least I have closed the
> > previous searcher and oldReader before creating new ones. However, I
> don't
> > know if it is a good practice to do so.
>
> You get the entrySet from the Map, you then iterate over its
> Map.Entry, then you replace in your original map some entries (the
> ones that are opened).  So, you are modifying a Java collection while
> iterating over elements from its Set view... I just don't know if
> that's safe (anyone?).  Would be good to instrument/debug and confirm
> that the precise IndexReader that's searching the Directory your
> IndexWriter just committed to, is in fact reopened.
>
> Mike
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message