lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Hani Suleiman <h...@formicary.net>
Subject Re: Testcase failure on OSX
Date Tue, 30 Sep 2003 22:38:43 GMT
Aha, yeah, System.currentTimeMillis would definitely be of much higher 
resolution.

I know the problem also intermittently affects linux, as I've seen it 
happen on there too (but a lot more rarely).

I think it's a feature of the antiquated HFS mac filesystem that OSX 
yses by default, rather than the OS itself. Adding a long timestamp to 
the segments file would fix this everywhere I think, since the worst 
resolution timers out there are in the region of 10ms, which isn't so 
bad.

I can work on that (pretty big, I suspect) patch if everyone else is 
too busy, since I'm quite looking forward to all tests passing on all 
platforms. I can verify on windows, linux, solaris, and OSX. Of course 
it'd be better if someone a bit more experienced with the codebase were 
to handle it ;)

On Tuesday, September 30, 2003, at 06:37 PM, Dmitry Serebrennikov wrote:

> Hani Suleiman wrote:
>
>> The count for FSDirectory on OSX is around 700-800. The problem is 
>> that  the filesystem timestamps on OSX has abysmal resolution 
>> (nothing finer  than 1 whole second!)
>
> Wow! Write once, debug everywhere...
> Is this an OSX "feature" or is this a feature of a particular disk / 
> filesystem / driver / whatnot that is in use in the current run?
> If this is unavoidable on OSX, perhaps other platforms are also 
> affected? Does anyone know the timestamp resolution on Wintel, Lintel, 
> Solaris, etc?
>
> Finally, what if we allocate a "long" in the beginning of the files 
> that need to be touched (like the segments and other files) and use it 
> to store the full System.currentTimeMillis timestamp? Does this method 
> return a more granular number on OSX? This would be a big change, but 
> it would work the same on all platforms and will not have a slowdown.
>
>>
>>
>> So the touch method will spin its wheels for anything up to a second  
>> before we've registered an actual 'touch'.
>>
>> This isn't so bad as files aren't touch that often, but it's a lot 
>> more  problematic in the case of SegmentInfos.write(), where a new 
>> file is  created, then copied over the old one. The  
>> testDeleteReaderWriteConflict testcase fails due to this. The old 
>> file  and the new have the same timestamp, and so the staleness check 
>> fails.  How should this be fixed?
>>
>> Also, won't the fix for FSDirectory mean there's a window where other 
>>  threads could end up with bad info, if we're taking up to a second 
>> to  touch a file?
>
> I don't know off hand, but I believe there is file locking in place to 
> prevent other threads from getting into this code until the update 
> operation is completed, and it will not be completed until the touch 
> finishes. This still sucks with a 1-second blocking...
>
> Dmitry.
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: lucene-dev-unsubscribe@jakarta.apache.org
> For additional commands, e-mail: lucene-dev-help@jakarta.apache.org
>
>


Mime
View raw message