lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Michael McCandless <luc...@mikemccandless.com>
Subject Re: Refreshing RAMDirectory
Date Wed, 12 Dec 2007 10:36:04 GMT

Ruslan Sivak wrote:

> Michael McCandless wrote:
>>
>> Ruslan Sivak wrote:
>>
>>> I have an index of about 10mb.  Since it's so small, I would like  
>>> to keep it loaded in memory, and reload it about every minute or  
>>> so, assuming that it has changed on disk.  I have the following  
>>> code, which works, except it doesn't reload the changes.
>>> protected String indexName;
>>> protected IndexReader reader;
>>> private long lastCheck=0;
>>> ...
>>> protected IndexReader getReader() throws CorruptIndexException,  
>>> IOException
>>>    {
>>>        if (reader==null || System.currentTimeMillis() > lastCheck 
>>> +60000)
>>>        {
>>>            lastCheck=System.currentTimeMillis();
>>>            if (reader==null || !reader.isCurrent())
>>>            {
>>>                if (reader!=null)
>>>                    reader.close();
>>>                              Directory dir = new RAMDirectory 
>>> (indexName);
>>>                reader = IndexReader.open(dir);
>>>                searcher = new IndexSearcher(reader);
>>>            }
>>>        }
>>>        return reader;
>>> }
>>>
>>>
>>> Apparently reader.isCurrent() won't tell you if the underlying  
>>> FSDirectory has changed.
>>
>> That's right: your reader is only searching the RAMDirectory; it  
>> has no idea that your RAMDirectory was copied from an FSDirectory  
>> that has now changed.  (That ctor for RAMDirectory makes a full  
>> copy of what's currently in the FSDirectory and thereafter  
>> maintains no link to that FSDirectory).
>>
>>> I also had the following code before:
>>> instead of
>>> if (reader==null || !reader.isCurrent())
>>> I had
>>> if (reader==null || reader.getVersion() !=  
>>> IndexReader.getCurrentVersion(indexName))
>>
>> That 2nd line seems like it should have worked.  What version of  
>> Lucene are you using?  Are you really sure it's not showing the  
>> changes?  Can you print the two versions?  Every commit to the  
>> index (by IndexWriter) should increment that version number.
>>
> The 2nd line was working fine, however I was getting errors in  
> other places saying that the indexReader is closed.

Can you restructure your code, such that you open a new reader  
without first closing the old one, and then only once the open is  
complete, you swap the new reader in as "reader", wait for threads to  
finish using the old reader, then call close on the old one?

>>> I was getting a bunch of this indexreader is closed errors, and  
>>> I'm not sure why there's no method like reader.isClosed().
>>
>> That's spooky: can you explain why you're accidentally using a  
>> closed reader?  Your code above seems to replace reader after  
>> closing it.  Are there other threads that are using the reader  
>> while you are doing this re-opening?
>>
> There could be other threads using this, and there are other places  
> in the code that open and close readers.  My main problem I guess  
> is that I can't tell when a reader is closed.  Is there some method  
> I can use?  I basically want to do something like this.
> if (reader==null || reader.isClosed || reader.getVersion() !=  
> IndexReader.getCurrentVersion(indexName))

There is currently no way to ask a reader if it's closed.  I suppose  
you could do something like:

   try {
     version = reader.getVersion();
     isClosed = false;
   } catch (AlreadyClosedException e) {
     isClosed = true;
   }

However this is somewhat bad form.  It's better to restructure your  
code such that it's not possible to accidentally use a reader you had  
closed.

> Is reader threadsafe?  Should each invocation open it's own reader?

Reader is definitely thread safe, so you should share 1 reader across  
all threads.  You just need to take care in your app to not close a  
reader if other threads are still using it or will continue using it.

Mike

---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org


Mime
View raw message