lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Erick Erickson" <erickerick...@gmail.com>
Subject Re: Refreshing RAMDirectory
Date Wed, 12 Dec 2007 14:33:58 GMT
Even if you could tell a reader is closed, you'd wind up with
unmaintainable code. I envision you have a bunch of places
where you'd do something like

if (reader.isClosed()) {
     reader = create a new reader.
}

But practically, you'd be opening a new reader someplace,
closing it someplace else, opening it in another place......

This just leads to maintenance nightmares. For instance, how
could you determine what the state of a particular reader was
when trying to figure out why your searches didn't work if you don't
have a clue where/when it was opened?

Perhaps the easiest thing to do if you can't restructure your code
as Michael suggested is just employ a singleton pattern to give you
complete control over when/where a reader is opened.....

Best
Erick

On Dec 12, 2007 5:36 AM, Michael McCandless <lucene@mikemccandless.com>
wrote:

>
> Ruslan Sivak wrote:
>
> > Michael McCandless wrote:
> >>
> >> Ruslan Sivak wrote:
> >>
> >>> I have an index of about 10mb.  Since it's so small, I would like
> >>> to keep it loaded in memory, and reload it about every minute or
> >>> so, assuming that it has changed on disk.  I have the following
> >>> code, which works, except it doesn't reload the changes.
> >>> protected String indexName;
> >>> protected IndexReader reader;
> >>> private long lastCheck=0;
> >>> ...
> >>> protected IndexReader getReader() throws CorruptIndexException,
> >>> IOException
> >>>    {
> >>>        if (reader==null || System.currentTimeMillis() > lastCheck
> >>> +60000)
> >>>        {
> >>>            lastCheck=System.currentTimeMillis();
> >>>            if (reader==null || !reader.isCurrent())
> >>>            {
> >>>                if (reader!=null)
> >>>                    reader.close();
> >>>                              Directory dir = new RAMDirectory
> >>> (indexName);
> >>>                reader = IndexReader.open(dir);
> >>>                searcher = new IndexSearcher(reader);
> >>>            }
> >>>        }
> >>>        return reader;
> >>> }
> >>>
> >>>
> >>> Apparently reader.isCurrent() won't tell you if the underlying
> >>> FSDirectory has changed.
> >>
> >> That's right: your reader is only searching the RAMDirectory; it
> >> has no idea that your RAMDirectory was copied from an FSDirectory
> >> that has now changed.  (That ctor for RAMDirectory makes a full
> >> copy of what's currently in the FSDirectory and thereafter
> >> maintains no link to that FSDirectory).
> >>
> >>> I also had the following code before:
> >>> instead of
> >>> if (reader==null || !reader.isCurrent())
> >>> I had
> >>> if (reader==null || reader.getVersion() !=
> >>> IndexReader.getCurrentVersion(indexName))
> >>
> >> That 2nd line seems like it should have worked.  What version of
> >> Lucene are you using?  Are you really sure it's not showing the
> >> changes?  Can you print the two versions?  Every commit to the
> >> index (by IndexWriter) should increment that version number.
> >>
> > The 2nd line was working fine, however I was getting errors in
> > other places saying that the indexReader is closed.
>
> Can you restructure your code, such that you open a new reader
> without first closing the old one, and then only once the open is
> complete, you swap the new reader in as "reader", wait for threads to
> finish using the old reader, then call close on the old one?
>
> >>> I was getting a bunch of this indexreader is closed errors, and
> >>> I'm not sure why there's no method like reader.isClosed().
> >>
> >> That's spooky: can you explain why you're accidentally using a
> >> closed reader?  Your code above seems to replace reader after
> >> closing it.  Are there other threads that are using the reader
> >> while you are doing this re-opening?
> >>
> > There could be other threads using this, and there are other places
> > in the code that open and close readers.  My main problem I guess
> > is that I can't tell when a reader is closed.  Is there some method
> > I can use?  I basically want to do something like this.
> > if (reader==null || reader.isClosed || reader.getVersion() !=
> > IndexReader.getCurrentVersion(indexName))
>
> There is currently no way to ask a reader if it's closed.  I suppose
> you could do something like:
>
>   try {
>     version = reader.getVersion();
>     isClosed = false;
>   } catch (AlreadyClosedException e) {
>     isClosed = true;
>   }
>
> However this is somewhat bad form.  It's better to restructure your
> code such that it's not possible to accidentally use a reader you had
> closed.
>
> > Is reader threadsafe?  Should each invocation open it's own reader?
>
> Reader is definitely thread safe, so you should share 1 reader across
> all threads.  You just need to take care in your app to not close a
> reader if other threads are still using it or will continue using it.
>
> Mike
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message