lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Chris and Helen Bamford <>
Subject DocValues and SearcherManager
Date Fri, 20 Oct 2017 09:28:16 GMT

I am using Lucene 4.10.3 and have a problem retrieving a DocValue field 
of a document using SearcherManager after I have updated a stored field 

The document has two key values: 'state' (stored Field) and 'id' 

After the document is indexed, it undergoes the following chain of events:

  - it is retrieved from the index by 'state' (using a Searcher obtained 
by SearcherManager.maybeRefresh() & searcherManager.acquire())
  - the 'state' field's value is changed and the document is updated 
using the IndexWriter from the SearcherManager 
(indexWriter.updateDocument(Term, Document))

This all works fine.

The problem comes when I want to match on the docValue 'id' reusing the 
same Searcher (SearcherManager.maybeRefresh() + 
searcherManager.acquire()), which does not work.

I'm no expert but it seems that when the document is retrieved by 
'state' it has only stored fields in the list, so when updated it ends 
up calling FieldInfos.addOrUpdate that discards FieldInfo of the 
docValue field 'myId' from the list. Afterwards it is impossible to 
retrieve the docValue using the same searcher 
(searcherManager.maybeRefresh() + searcherManager.acquire()).

If a new reader is obtained the docValue match/update is possible but 
this is a performance critical piece of code and I was hoping to reuse 
the same SearcherManager.

The unit test here shows the problem:

public class SearcherManagerFailureTest {
     private static final String indexPath = "/tmp/mytestindex";

     private IndexWriter indexWriter;
     private SearcherManager searcherManager;
     public Directory directory;

     public void beforeTest() throws Exception {
         // Setup
         directory = File(indexPath));
         IndexWriterConfig idxCfg = new 
IndexWriterConfig(Version.LUCENE_4_10_3, new WhitespaceAnalyzer());
         indexWriter = new IndexWriter(directory, idxCfg);
         searcherManager = new SearcherManager(indexWriter, true, new 

     public void afterTest() throws Exception {
         if (indexWriter != null) {

         if (searcherManager != null) {

         if (directory != null) {

         for(File file: new File(indexPath).listFiles())
             if (!file.isDirectory())

     public void TestSearcherManagerFails() throws Exception{

         Document doc = new Document();
         FieldType ft = new FieldType(TextField.TYPE_STORED);
         doc.add(new Field("docId",  "doc1",   ft));
         doc.add(new Field("state",   "added",   ft));
         doc.add(new BinaryDocValuesField("id", new BytesRef("first")));

         //Search by state
         IndexSearcher searcher = searcherManager.acquire();
         TopDocs topDocs = TermQuery(new 
Term("state", "added")), null, 1);
         Document indexedDoc = searcher.doc(topDocs.scoreDocs[0].doc);

         //Update document
         String docId = indexedDoc.get("docId");
         Term term = new Term("docId", docId);
         Field stateField = (Field) indexedDoc.getField("state");
         indexWriter.updateDocument(term, indexedDoc);

         //Try get docValue
         IndexSearcher newSearcher = searcherManager.acquire();

         BinaryDocValues docValues = 
MultiDocValues.getBinaryValues(newSearcher.getIndexReader(), "id");
         Assert.assertEquals(null, docValues);

         Directory newDirectory = File(indexPath));
         BinaryDocValues docValues2 = 
MultiDocValues.getBinaryValues(, "id");
         Assert.assertNotSame(null, docValues2);

         if(newDirectory != null){

Can anyone advise?


- Chris

  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message