lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Grant Ingersoll <>
Subject Re: Semantics of a closed IndexInput
Date Tue, 04 Apr 2006 12:56:06 GMT
OK.  They should not be used, but we have no way of determining if a 
IndexInput is actually closed, right?  At least Lucene does not track 
it.  I run into this issue with Lazy Loading.  I can still access the 
value of a lazy field after the fieldsStream IndexInput is closed _if_ I 
have not made my thread local clone yet, but do so after closing the 

Code looks like (for example):
//This is my new, private Field implementation, which is an inner class 
of FieldsReader
private class LazyField implements Fieldable
    public String stringValue() {
      if (fieldsData == null)
        IndexInput localFieldsStream = (IndexInput) fieldsStreamTL.get();
        if (localFieldsStream == null) {
          localFieldsStream = (IndexInput) fieldsStream.clone();
        try {
          //read in chars b/c we already know the length we need to read
          if (chars == null || toRead > chars.length)
            chars = new char[toRead];
          localFieldsStream.readChars(chars, 0, toRead);
          fieldsData = new String(chars, 0, 
        } catch (IOException e) {
          throw new FieldReaderException(e);
      return fieldsData instanceof String ? (String)fieldsData : null;

fieldsStreamTL is a private static ThreadLocal belonging to 
FieldsReader.  In FieldsReader.close() I do close the associated 
ThreadsLocal clone if it exists, as in:
final void close() throws IOException {
    IndexInput localFieldsStream = (IndexInput) fieldsStreamTL.get();
    if (localFieldsStream != null) {

However if the application does something like:   
public void testLazyClosedReader() throws Exception {
    assertTrue(dir != null);
    assertTrue(fieldInfos != null);
    FieldsReader reader = new FieldsReader(dir, "test", fieldInfos);
    assertTrue(reader != null);
    assertTrue(reader.size() == 1);
    Set lazyFieldNames = new HashSet();
    //new String[]{DocHelper.LARGE_LAZY_FIELD_KEY, 
    Document doc = reader.doc(0, lazyFieldNames);
    assertTrue("doc is null and it shouldn't be", doc != null);
    Fieldable field = doc.getField(DocHelper.LAZY_FIELD_KEY);
    assertTrue("field is null and it shouldn't be", field != null);
    String value;
    try {
      value = field.stringValue();
      assertTrue("Should not be able to read value: " + value + " since 
the reader is closed", false);
    } catch (Exception e) {
    field = doc.getField(DocHelper.TEXT_FIELD_1_KEY);
    assertTrue("field is null and it shouldn't be", field != null);
    value = field.stringValue();
    assertTrue("value is null and it shouldn't be", value != null);
    assertTrue(value + " is not equal to " + DocHelper.FIELD_1_TEXT, 
value.equals(DocHelper.FIELD_1_TEXT) == true);

This test fails, and I don't think it should.  The assert in the try 
block is activated.  What am I missing?

What do you think of adding an boolean isClosed() onto IndexInput so 
that we can test for closure.  I suppose this could also be added to 
anything that has close() as a method.

Doug Cutting wrote:
> Grant Ingersoll wrote:
>> Should it be the case that you can clone a closed IndexInput and get 
>> a valid object that is capable of reading?  B/c this is what I am 
>> seeing in my Lazy implementation (note, it seems to work fine...)  I 
>> am just not sure if it should work or if it is a bug.
> Cloned IndexInputs are assumed to share resources with the original. 
> Lucene closes all original IndexInputs when the IndexReader is closed. 
> Cloned IndexInputs should not be used after the IndexReader is closed. 
> I don't think we bother to explicitly close clones (as they are 
> frequently cached in ThreadLocals, etc.).
> Doug
> ---------------------------------------------------------------------
> To unsubscribe, e-mail:
> For additional commands, e-mail:


Grant Ingersoll 
Sr. Software Engineer 
Center for Natural Language Processing 
Syracuse University 
School of Information Studies 
335 Hinds Hall 
Syracuse, NY 13244 
Voice:  315-443-5484 
Fax: 315-443-6886 

To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message