lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Scott Ganyo <scott.ga...@eTapestry.com>
Subject RE: IndexReader.lastModified() not correct
Date Wed, 07 Aug 2002 17:40:42 GMT
I lock the index because I copied the code from the open() method... and
because it seemed to be the right thing to do.  I was concerned that the
index files not change during my iterating over them.  For example, if the
index segments were merged into a new one between when I read the "segments"
file and when I iterated over the segment's deleted files, I wouldn't
necessarily get the right answer.  On the other hand, I could check the last
modified date on the "segments" file *after* checking the individual
segments.  Maybe I'll do that instead.

At any rate, though, you're right.  It isn't truly safe unless the lock is
held during the entire process...

Scott

P.S. I don't know if this could have caused your Exception.  I actually
can't think of a scenario off-hand where you would get that Exception except
in a corrupt index...  is your index intact?

> -----Original Message-----
> From: Halácsy Péter [mailto:halacsy.peter@axelero.com]
> Sent: Wednesday, August 07, 2002 12:23 PM
> To: Lucene Developers List
> Subject: RE: IndexReader.lastModified() not correct
> 
> 
> Scott,
> why do you lock the index while iterating del files? 
> 
> if you don't lock it, you can calculate wrong time but has it 
> effect on caching? one can modify the index between checking 
> the lastModified value and creating new searcher.
> 
> peter
> 
> > -----Original Message-----
> > From: Scott Ganyo [mailto:scott.ganyo@eTapestry.com]
> > Sent: Wednesday, August 07, 2002 6:32 PM
> > To: 'Lucene Developers List'
> > Subject: RE: IndexReader.lastModified() not correct
> > 
> > 
> > Ok.  I've dug through all the code and although this isn't 
> > too pretty, it
> > looks to my untrained eye that this is probably the best way 
> > to implement a
> > corrected version that takes into consideration the deleted list.
> > Basically, it checks the "segments" file as before, then checks the
> > "segment.del" file in each segment, and takes the last time 
> > any of those
> > files were modified:
> > 
> >   public static long lastModified(final Directory directory) throws
> > IOException {
> >     synchronized (directory) {			  // 
> > in- & inter-process
> > sync
> >       Long lastMod = (Long)new 
> > Lock.With(directory.makeLock("commit.lock"))
> > {
> >         public Object doBody() throws IOException {
> >           long last = directory.fileModified("segments"); // 
> > last add time
> >           SegmentInfos infos = new SegmentInfos();
> >           infos.read(directory);
> >           for (int i = 0; i < infos.size(); i++) {
> >             String delFile = infos.info(i).name + ".del";
> >             if (directory.fileExists(delFile)) {
> >                 long lastDel = 
> > directory.fileModified(delFile); // last
> > delete time (per segment)
> >                 if (lastDel > last) last = lastDel;
> >             }
> >           }
> >           return new Long(last);
> >         }
> >       }.run();
> >       return lastMod.longValue();
> >     }
> >   }
> > 
> > Please let me know if I've missed any shortcuts or if you 
> > know of a better
> > way...
> > 
> > Scott
> > 
> > > -----Original Message-----
> > > From: Scott Ganyo [mailto:scott.ganyo@eTapestry.com]
> > > Sent: Wednesday, August 07, 2002 9:27 AM
> > > To: Lucene-Dev (E-mail)
> > > Subject: IndexReader.lastModified() not correct
> > > 
> > > 
> > > The current implementation of IndexReader.lastModified() does 
> > > not return the
> > > results I am expecting.  Here is the implementation:
> > > 
> > >   /** Returns the time the index in the named directory was 
> > > last modified.
> > > */
> > >   public static long lastModified(File directory) throws 
> > IOException {
> > >     return FSDirectory.fileModified(directory, "segments");
> > >   }
> > > 
> > > The problem is that the "segments" file is apparently not 
> > > updated when just
> > > doing a delete from an index.  This is causing me problems 
> > > because I am
> > > attempting to rely on on the lastModified() command for 
> > IndexSearcher
> > > caching.  The only solution that I have thought of so far 
> > > (without changing
> > > other parts of Lucene) is to make the lastModified() command 
> > > look at all the
> > > files in the last segment for the last modified date.
> > > 
> > > Thoughts?
> > > 
> > > Thanks,
> > > Scott
> > > 
> > 
> 
> --
> To unsubscribe, e-mail:   
> <mailto:lucene-dev-unsubscribe@jakarta.apache.org>
> For additional commands, e-mail: 
> <mailto:lucene-dev-help@jakarta.apache.org>
> 

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message