lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Robert Engels" <>
Subject RE: GData, updateable IndexSearcher
Date Mon, 01 May 2006 18:49:41 GMT

It uses subclasses and instanceof which is sort of "hackish" - to do it
correctly requires changes to the base classes.

-----Original Message-----
From: jason rutherglen []
Sent: Monday, May 01, 2006 1:43 PM
Subject: Re: GData, updateable IndexSearcher

Can you post your code?

----- Original Message ----
From: Robert Engels <>
To:; jason rutherglen <>
Sent: Monday, May 1, 2006 11:33:06 AM
Subject: RE: GData, updateable IndexSearcher

fyi, using my reopen(0 implementation (which rereads the deletions)

on a 135mb index, with 5000 iterations

open & close time using new reader = 585609
open & close time using reopen = 27422

Almost 20x faster. Important in a highly interactive/incremental updating

-----Original Message-----
From: jason rutherglen []
Sent: Monday, May 01, 2006 1:24 PM
Subject: Re: GData, updateable IndexSearcher

I wanted to post a quick hack to see if it is along the correct lines.  A
few of the questions regard whether to resuse existing MultiReaders or
simply strip out only the SegmentReaders.  I do a compare on the segment
name and made it public.  Thanks!

public static IndexReader reopen(IndexReader indexReader) throws IOException
    if (indexReader instanceof MultiReader) {
      MultiReader multiReader = (MultiReader)indexReader;

      SegmentInfos segmentInfos = new SegmentInfos();;
      if (segmentInfos.size() == 1) {          // index is optimized
        return SegmentReader.get(segmentInfos,, false);

      IndexReader[] existingIndexReaders = multiReader.getSubReaders();
      // now go through and compare the segment readers
      Map<String,SegmentReader> existingSegmentMap = new
      getSegmentReaders(existingIndexReaders, existingSegmentMap);

      Map<String,SegmentInfo> newSegmentInfosMap = new

      List<SegmentReader> newSegmentReaders = new

      Iterator segmentInfosIterator = segmentInfos.iterator();
      while (segmentInfosIterator.hasNext()) {
        SegmentInfo segmentInfo = (SegmentInfo);

        if (!existingSegmentMap.containsKey( {
          // it's new
          SegmentReader newSegmentReader = SegmentReader.get(segmentInfo);
      List<IndexReader> allSegmentReaders = new ArrayList<IndexReader>();

      return new MultiReader(, segmentInfos, false,
(IndexReader[])allSegmentReaders.toArray(new IndexReader[0]));
    throw new RuntimeException("indexReader not supported at this time");

  public static void getSegmentReaders(IndexReader[] indexReaders,
Map<String,SegmentReader> map) {
    for (int x=0; x < indexReaders.length; x++) {
      if (indexReaders[x] instanceof MultiReader) {
        MultiReader multiReader = (MultiReader)indexReaders[x];
        IndexReader[] subReaders = multiReader.getSubReaders();
        getSegmentReaders(subReaders, map);
      } else if (indexReaders[x] instanceof SegmentReader) {
        SegmentReader segmentReader = (SegmentReader)indexReaders[x];
        map.put(segmentReader.segment, segmentReader);

View raw message