couchdb-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Juhani Ränkimies <juh...@juranki.com>
Subject Re: Windows 0.11 snapshot
Date Sat, 27 Feb 2010 00:17:07 GMT
On Sat, Feb 27, 2010 at 1:00 AM, Mark Hammond <skippy.hammond@gmail.com> wrote:

> Regarding the file versioning - I'm sorry that we haven't kept you in the
> loop a little better, but with Damien's guidance we think we have found a
> better strategy for this.

nb, it's great that the issue gets attention.

>
> The root of this strategy comes from a realization that if a file is opened
> on windows with FILE_SHARE_DELETE, the file can be deleted *or renamed*
> while it is open.  One limitation is that a file of the same name can not be
> re-created while the old one still has handles open (the 'deleted but still
> open' file still appears in directory listings until the handle is closed,
> for example)
>
> Given this, what we can do is something like:
>
> * Arrange for erlang to be able to open the DB and view files with this
> flag.
> * Instead of deleting a file before replacing it, we first rename the file
> to a unique name (ie, based on a UUID) in a special directory.
> * As couch starts up, attempt to delete any old files in this special
> directory.  In theory, no such files should exist - the OS should take care
> of actually removing any such files even if erlang crashes.
>
> The end result of this is that things can be made to work with a lot less
> friction than the 'file versioning' scheme.  I've a patch to couchdb that
> works when used with a patch to erlang to open *all* files with that flag.
>  The next step down this path seems to be to create an erlang 'port driver'
> for disk IO and use this instead of the builtin erlang file objects.  We've
> identified a 'bfile' erlang port driver we may be able to fork and adapt to
> our needs - using our own port driver would also allow optimizations to the
> file IO for non-windows platforms (eg, indicate to the OS that certain files
> should not get OS-level caching, etc)
>

Sounds like a good plan. I noticed Jan just filed a proposal to
erlang-patches for adding FILE_SHARE_DELETE flag to CreateFile in
erts.

> Unfortunately, none of this has been discussed anywhere formal - much of it
> was on IRC - so I apologize if this is the first you heard of it. But
> whatever strategy we wind up with for this is almost certainly not going to
> land and be tested enough to make 0.11.

Any idea when it'll land to trunk?
I could use a rough estimate to decide if I should temporarily deploy
the file versioning patch.


cheers,
-juhani

Mime
View raw message