lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Dmitry Serebrennikov <dmit...@earthlink.net>
Subject Re: 1.3 RC2 / timestamp
Date Wed, 22 Oct 2003 17:30:49 GMT
goller@detego-software.de wrote:

>Unfortunately, I can´t do anything on the timestamp problem before 
>Friday since I am at a customer´s office. So RC2 has to come out 
>without my contribution, but I think that´s ok since it is only a RC. 
>
>I am not completely sure if including the new timestamp/version number 
>into the filename is more efficient than having it in the file. 
>If the version number is in the filename, do have have to search for 
>the file. I will not know the exact name, will I? 
>
>Christoph
>
>---------------------------------------------------------------------
>To unsubscribe, e-mail: lucene-dev-unsubscribe@jakarta.apache.org
>For additional commands, e-mail: lucene-dev-help@jakarta.apache.org
>
>
>  
>
Yes, plus we will have to expect multiple files with timestamped names 
present at the same time (to deal with possible race conditions). 
Listing a directory can be expensive when number of files in it is 
large, so during indexing with large marge factor this can be more 
expensive. However on an optimized index this may not be so bad. Another 
consideration is that updates of a contents of a file are not atomic 
while updates to the timestamp are (or they should be, damn it! :)   ).

I also agree that it is more expensive to read/update the contents of a 
file than it is to deal with the metadata (filesystem timestamp). How 
about these ideas for improving the situation:

- Use the filesystem timestamp AND a timestamp saved inside a file. The 
hope here is that in most cases filesystem check will be all that is 
needed, but when this is not enough the file will be opened and read 
automatically, resolving any umbiguity. Filesystem locking will have to 
be relied upon to make sure nobody reads a timestamp while someone else 
is writing it.

- Store timestamp in a file that is alread kept open. This removes the 
issue of having to open and read an extra file. We are just talking 
about doing a seek to beginning or end of a file and then 
reading/writing the value there. Again, filesystem locking will have to 
ensure that there is no conflict between readers and writers.




---------------------------------------------------------------------
To unsubscribe, e-mail: lucene-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: lucene-dev-help@jakarta.apache.org


Mime
View raw message