lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hani Suleiman" <h...@formicary.net>
Subject Re: Testcase failure on OSX
Date Tue, 30 Sep 2003 17:22:05 GMT
Yep, will do so tonight. I saw the code in RAMDirectory and couldn't
figure out why the same logic wasn't in DSDirectory.

I'll test on OSX and linux and report back.

Dmitry Serebrennikov said:
> Sounds like the same error that I fixed in the RAMDirectory last week. I
>  didn't think that FSDirectory would have this problem because I thought
>  that the OS filesystem would take care of this. Apparently it does not.
>
> Hani, please take a look at the RAMDirectory touchFile method (line
> 148). It's basically the same fix you proposed except that it has some
> commented out monitoring code and it uses Thread.sleep(0,1) to release
> the CPU while waiting. In my measurements I was seeing 5-10 loops with
> the Thread.sleep call (regardless of the actual sleep duration that I
> specified), and around 10000 loops without a sleep call. But of course I
>  was on a different hardware/OS combo and probably a different VM.
>
> Hani, will you be able to try a variant of this fix for the FSDirectory?
> I'm currious to see what kinds of results will observed on OS X.
>
> Thanks.
> Dmitry.
>
>
> Hani Suleiman wrote:
>
>> Ok, I figured out what's going on.
>>
>> I suspect this will only manifest itself on faster machines (I'm on a
>>  dual 1.42ghz G4)
>>
>> What happens basically is that the the timer resolution on
>> file.lastModified isn't accurate enough, so when a file is touched,
>> it  returns the same value for lastModified. This in turn causes the
>> segment staleness checks in IndexReader to fail (around line 276,
>> lastModified(directory)==segmentInfosAge), hence the testcase fails.
>>
>> I believe this is a bug, but I can't come up with a satisfactory
>> workaround. My solution is to modify FSDirectory.touchFile(String
>> name)  to have the following block;
>>
>>     long oldModified = file.lastModified();
>>     do
>>     {
>>       file.setLastModified(System.currentTimeMillis());
>>     }
>>     while(oldModified==file.lastModified());
>>
>> This ensures that a touch ensures the file is considered 'fresher'
>> that  it was before, which makes everything happy. It's a bit of a
>> hack  though, so I'm not sure of how you guys would prefer handling
>> this,  take the performance hit with the extra lastModified checks, if
>>  make it  an even bigger hack by checking for OSX, or something far
>> more clever  and not detrimental to performance ;)
>>
>> Hani
>>
>> On Tuesday, September 30, 2003, at 12:42 AM, Hani Suleiman wrote:
>>
>>> here you go, happens to me on linux too (with JDK 1.4.0)
>>>
>>> OSX using latest 1.4.1
>>>
>>>    [junit] ------------- ---------------- ---------------
>>>     [junit] Testsuite: org.apache.lucene.index.TestIndexReader
>>>     [junit] Tests run: 5, Failures: 2, Errors: 0, Time elapsed:
>>> 2.883  sec
>>>
>>>     [junit] Testcase:
>>> testDeleteReaderWriterConflict(org.apache.lucene.index.TestIndexReader)
>>>  :  FAILED
>>>     [junit] Delete allowed on an index reader with stale segment
>>> information
>>>     [junit] junit.framework.AssertionFailedError: Delete allowed on
>>> an  index reader with stale segment information
>>>     [junit]     at
>>> org.apache.lucene.index.TestIndexReader.testDeleteReaderWriterConflict(
>>>  TestIndexReader.java:271)
>>>     [junit]     at
>>> sun.reflect.NativeMethodAccessorImpl.invoke0(Native  Method)
>>>     [junit]     at
>>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.ja
>>>  va:39)
>>>     [junit]     at
>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccesso
>>>  rImpl.java:25)
>>>
>>>
>>>     [junit] Testcase:
>>> testDeleteReaderReaderConflict(org.apache.lucene.index.TestIndexReader)
>>>  :  FAILED
>>>     [junit] Delete allowed from a stale index reader
>>>     [junit] junit.framework.AssertionFailedError: Delete allowed
>>> from  a stale index reader
>>>     [junit]     at
>>> org.apache.lucene.index.TestIndexReader.testDeleteReaderReaderConflict(
>>>  TestIndexReader.java:397)
>>>     [junit]     at
>>> sun.reflect.NativeMethodAccessorImpl.invoke0(Native  Method)
>>>     [junit]     at
>>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.ja
>>>  va:39)
>>>     [junit]     at
>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccesso
>>>  rImpl.java:25)
>>>
>>>
>>>
>>> On Tuesday, September 30, 2003, at 12:37 AM, Erik Hatcher wrote:
>>>
>>>> Hani,
>>>>
>>>> I don't see this test failure on OS X.  Thanks to your patch on the
>>>>  RussianAnalyzer, all my tests now successfully pass.  Could you
>>>> give  us the stack trace for the problem you're seeing?
>>>>
>>>>     Erik
>>>>
>>>>
>>>> On Monday, September 29, 2003, at 11:28  PM, Hani Suleiman wrote:
>>>>
>>>>> I'm seeing a fairly disturbing testcase failure on OSX in
>>>>> TestIndexReader.testDeleteReaderWriterConflict and
>>>>> TestIndexReader.testDeleteReaderReaderConflict
>>>>>
>>>>> This only happens on OSX though, is this a known issue? Any have
>>>>> an  ideas of what sort of thing is going wrong before I dig in
>>>>> further?
>>>>>
>>>>> Hani
>>>>>
>>>>>
>>>>> ---------------------------------------------------------------------
>>>>> To unsubscribe, e-mail: lucene-dev-unsubscribe@jakarta.apache.org
>>>>> For additional commands, e-mail: lucene-dev-help@jakarta.apache.org
>>>>
>>>>
>>>>
>>>> ---------------------------------------------------------------------
>>>> To unsubscribe, e-mail: lucene-dev-unsubscribe@jakarta.apache.org
>>>> For additional commands, e-mail: lucene-dev-help@jakarta.apache.org
>>>>
>>>>
>>>
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: lucene-dev-unsubscribe@jakarta.apache.org For
>>> additional commands, e-mail: lucene-dev-help@jakarta.apache.org
>>>
>>>
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: lucene-dev-unsubscribe@jakarta.apache.org For
>> additional commands, e-mail: lucene-dev-help@jakarta.apache.org
>>
>>
>
>
>
> --------------------------------------------------------------------- To
> unsubscribe, e-mail: lucene-dev-unsubscribe@jakarta.apache.org
> For additional commands, e-mail: lucene-dev-help@jakarta.apache.org





Mime
View raw message