lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Scott Ganyo <scott.ga...@eTapestry.com>
Subject RE: IndexReader.lastModified() not correct
Date Wed, 07 Aug 2002 17:47:16 GMT
Ok.  Here's a new, simplified version that doesn't lock the index:

  /** Returns the time the index in this directory was last modified. */
  public static long lastModified(final Directory directory) throws
IOException {
    long last = 0;
    SegmentInfos infos = new SegmentInfos();
    infos.read(directory);
    for (int i = 0; i < infos.size(); i++) {
      String delFile = infos.info(i).name + ".del";
      if (directory.fileExists(delFile)) {
          long lastDel = directory.fileModified(delFile); // last delete
time (per segment)
          if (lastDel > last) last = lastDel;
      }
    }
    long lastAdd = directory.fileModified("segments"); // last add time
    return (lastAdd > last) ? lastAdd : last;
  }

Comments?

Scott

> -----Original Message-----
> From: Scott Ganyo [mailto:scott.ganyo@eTapestry.com]
> Sent: Wednesday, August 07, 2002 12:41 PM
> To: 'Lucene Developers List'
> Subject: RE: IndexReader.lastModified() not correct
> 
> 
> I lock the index because I copied the code from the open() 
> method... and
> because it seemed to be the right thing to do.  I was 
> concerned that the
> index files not change during my iterating over them.  For 
> example, if the
> index segments were merged into a new one between when I read 
> the "segments"
> file and when I iterated over the segment's deleted files, I wouldn't
> necessarily get the right answer.  On the other hand, I could 
> check the last
> modified date on the "segments" file *after* checking the individual
> segments.  Maybe I'll do that instead.
> 
> At any rate, though, you're right.  It isn't truly safe 
> unless the lock is
> held during the entire process...
> 
> Scott
> 
> P.S. I don't know if this could have caused your Exception.  
> I actually
> can't think of a scenario off-hand where you would get that 
> Exception except
> in a corrupt index...  is your index intact?
> 
> > -----Original Message-----
> > From: Halácsy Péter [mailto:halacsy.peter@axelero.com]
> > Sent: Wednesday, August 07, 2002 12:23 PM
> > To: Lucene Developers List
> > Subject: RE: IndexReader.lastModified() not correct
> > 
> > 
> > Scott,
> > why do you lock the index while iterating del files? 
> > 
> > if you don't lock it, you can calculate wrong time but has it 
> > effect on caching? one can modify the index between checking 
> > the lastModified value and creating new searcher.
> > 
> > peter
> > 
> > > -----Original Message-----
> > > From: Scott Ganyo [mailto:scott.ganyo@eTapestry.com]
> > > Sent: Wednesday, August 07, 2002 6:32 PM
> > > To: 'Lucene Developers List'
> > > Subject: RE: IndexReader.lastModified() not correct
> > > 
> > > 
> > > Ok.  I've dug through all the code and although this isn't 
> > > too pretty, it
> > > looks to my untrained eye that this is probably the best way 
> > > to implement a
> > > corrected version that takes into consideration the deleted list.
> > > Basically, it checks the "segments" file as before, then 
> checks the
> > > "segment.del" file in each segment, and takes the last time 
> > > any of those
> > > files were modified:
> > > 
> > >   public static long lastModified(final Directory 
> directory) throws
> > > IOException {
> > >     synchronized (directory) {			  // 
> > > in- & inter-process
> > > sync
> > >       Long lastMod = (Long)new 
> > > Lock.With(directory.makeLock("commit.lock"))
> > > {
> > >         public Object doBody() throws IOException {
> > >           long last = directory.fileModified("segments"); // 
> > > last add time
> > >           SegmentInfos infos = new SegmentInfos();
> > >           infos.read(directory);
> > >           for (int i = 0; i < infos.size(); i++) {
> > >             String delFile = infos.info(i).name + ".del";
> > >             if (directory.fileExists(delFile)) {
> > >                 long lastDel = 
> > > directory.fileModified(delFile); // last
> > > delete time (per segment)
> > >                 if (lastDel > last) last = lastDel;
> > >             }
> > >           }
> > >           return new Long(last);
> > >         }
> > >       }.run();
> > >       return lastMod.longValue();
> > >     }
> > >   }
> > > 
> > > Please let me know if I've missed any shortcuts or if you 
> > > know of a better
> > > way...
> > > 
> > > Scott
> > > 
> > > > -----Original Message-----
> > > > From: Scott Ganyo [mailto:scott.ganyo@eTapestry.com]
> > > > Sent: Wednesday, August 07, 2002 9:27 AM
> > > > To: Lucene-Dev (E-mail)
> > > > Subject: IndexReader.lastModified() not correct
> > > > 
> > > > 
> > > > The current implementation of IndexReader.lastModified() does 
> > > > not return the
> > > > results I am expecting.  Here is the implementation:
> > > > 
> > > >   /** Returns the time the index in the named directory was 
> > > > last modified.
> > > > */
> > > >   public static long lastModified(File directory) throws 
> > > IOException {
> > > >     return FSDirectory.fileModified(directory, "segments");
> > > >   }
> > > > 
> > > > The problem is that the "segments" file is apparently not 
> > > > updated when just
> > > > doing a delete from an index.  This is causing me problems 
> > > > because I am
> > > > attempting to rely on on the lastModified() command for 
> > > IndexSearcher
> > > > caching.  The only solution that I have thought of so far 
> > > > (without changing
> > > > other parts of Lucene) is to make the lastModified() command 
> > > > look at all the
> > > > files in the last segment for the last modified date.
> > > > 
> > > > Thoughts?
> > > > 
> > > > Thanks,
> > > > Scott
> > > > 
> > > 
> > 
> > --
> > To unsubscribe, e-mail:   
> > <mailto:lucene-dev-unsubscribe@jakarta.apache.org>
> > For additional commands, e-mail: 
> > <mailto:lucene-dev-help@jakarta.apache.org>
> > 
> 

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message