lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "yueyu lin" <popeye...@gmail.com>
Subject Multiple threads searching in Lucene and the synchronized issue. -- solution attached.
Date Tue, 09 May 2006 07:16:33 GMT
Hi,
  We're building a server based Lucene. When doing the multiple threads
performance test, we found a minor synchronized issue.

  We found if we were using 2 IndexSearcher, we would get 10% performance
benefit.
  But if we increased the number of IndexSearcher from 2, the performance
improvement became slight even worse.
  To find why it is so, we traced into the codes. Fortunately we got it.

  The class org.apache.lucene.index.TermInfosReader , as you know, every
IndexSearcher will have one TermInfosReader. Every query, one method in the
class must be called:
private synchronized void ensureIndexIsRead() throws IOException . Notice
it, it's a synchronized function to ensure if the index is readable.
Although, if it's initialized, it will return immediately, but it's still a
lock. So if we use two IndexSearcher, the opportunity to wait will decrease
a lot. In some conditions for an example, a busy system and the query is
relatively quick, the opportunity to wait will increase a lot.
  The attachement is our modification for that code. It might be helpful to
increase the concurrency performance while querying. Can you read it and
file that as a bug to fix in the next release?

  The main change is like below:
  If you accept it, the sysout can be removed :).
     ....
    private void ensureIndexIsRead() throws IOException {
    if (indexTerms != null)                       // index already read
      return;                                     // do nothing
    synchronized(this){
        System.out.println("Read index@--@");
        if(indexTerms != null){
            System.out.println("Someone read it.return-_-");
            return ;
        }
        readIndex ();
    }
  }

  private synchronized void readIndex() throws IOException{
      Term[] m_indexTerms = null;
      try {
          int indexSize = (int)indexEnum.size;        // otherwise read
index
          m_indexTerms = new Term[indexSize];
          indexInfos = new TermInfo[indexSize];
          indexPointers = new long[indexSize];

          for (int i = 0; indexEnum.next(); i++) {
            m_indexTerms[i] = indexEnum.term();
            indexInfos[i] = indexEnum.termInfo();
            indexPointers[i] = indexEnum.indexPointer;
          }
        } finally {
            indexEnum.close();
            indexEnum = null;
            indexTerms = m_indexTerms;
        }
  }
--
--
Yueyu Lin

Mime
View raw message