Hani Suleiman wrote:
> The count for FSDirectory on OSX is around 700-800. The problem is
> that the filesystem timestamps on OSX has abysmal resolution (nothing
> finer than 1 whole second!)
Wow! Write once, debug everywhere...
Is this an OSX "feature" or is this a feature of a particular disk /
filesystem / driver / whatnot that is in use in the current run?
If this is unavoidable on OSX, perhaps other platforms are also
affected? Does anyone know the timestamp resolution on Wintel, Lintel,
Solaris, etc?
Finally, what if we allocate a "long" in the beginning of the files that
need to be touched (like the segments and other files) and use it to
store the full System.currentTimeMillis timestamp? Does this method
return a more granular number on OSX? This would be a big change, but it
would work the same on all platforms and will not have a slowdown.
>
>
> So the touch method will spin its wheels for anything up to a second
> before we've registered an actual 'touch'.
>
> This isn't so bad as files aren't touch that often, but it's a lot
> more problematic in the case of SegmentInfos.write(), where a new
> file is created, then copied over the old one. The
> testDeleteReaderWriteConflict testcase fails due to this. The old
> file and the new have the same timestamp, and so the staleness check
> fails. How should this be fixed?
>
> Also, won't the fix for FSDirectory mean there's a window where other
> threads could end up with bad info, if we're taking up to a second to
> touch a file?
I don't know off hand, but I believe there is file locking in place to
prevent other threads from getting into this code until the update
operation is completed, and it will not be completed until the touch
finishes. This still sucks with a 1-second blocking...
Dmitry.
|