Mailing-List: contact dev-help@couchdb.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@couchdb.apache.org
Received-SPF: pass (nike.apache.org: domain of antony.blakey@gmail.com
 designates 209.85.220.163 as permitted sender)
DomainKey-Signature: a=rsa-sha1; c=nofws;
        d=gmail.com; s=gamma;
        h=message-id:from:to:in-reply-to:content-type
         :content-transfer-encoding:mime-version:subject:date:references
         :x-mailer;
        b=iObldmelLVhHW5O109EIfG+0HewM8ZX3jq/jmgsO86/i367Fk//W73njNIfdPrHixH
         y5uX0ifn0PegXoVsJDX6VQS2DxdGUQn+5IHdDmOXAnUZwCTIxp3291Bdfb5Dw2HmElCh
         97smO8hNA8X7TzSf76z4wivdyA/WTmiGxCUuE=
Message-Id: <1D8DDDAA-D5AF-48E1-9373-91E04E75FF0F@gmail.com>
From: Antony Blakey <antony.blakey@gmail.com>
To: dev@couchdb.apache.org
In-Reply-To: <9C5ADA1E-83C3-4CBF-B7B8-33CB31A4E084@apache.org>
Content-Type: text/plain; charset=US-ASCII; format=flowed; delsp=yes
Content-Transfer-Encoding: 7bit
Mime-Version: 1.0 (Apple Message framework v930.4)
Subject: Re: Tail Append Headers
Date: Wed, 20 May 2009 09:48:01 +0800
References: <0E57756C-BC55-4AC3-9367-3E1B86A2D69D@apache.org>
 <e282921e0905181643v30d9ca22r47a5df55121e18c4@mail.gmail.com>
 <9C5ADA1E-83C3-4CBF-B7B8-33CB31A4E084@apache.org>


On 20/05/2009, at 12:50 AM, Damien Katz wrote:

> Also, if performance generally turns out to be all around slower,  
> we'll have to discuss if the pure tail append change is actually  
> worth it. Maybe we can tail append headers with the old design too,  
> but they are only ever used when the front header is bad. The only  
> problem is, without implementing the current design, I don't know of  
> a workable way to find an valid header vs something that happens to  
> look like a couchdb file header, such as a couchdb file attached  
> inside a document in a live db, or an intentional attack.

Maybe you could do this by ensuring that:

a) headers are only on a certain boundary, even a small boundary such  
as 128 bytes, but other writes ignore alignment; and
b) headers include a private (unexposed) UUID of the database (stored  
in an immutable file prefix); and
c) headers include the write offset; and
d) headers include magic (although maybe the private UUID is enough).

e.g <magic><private UUID><write offset>

Point a reduces the search space - maybe that's not necessary given  
it's a serious failure if you have to use it; and
Points b and c make an intentional attack virtually impossible.
Points a and c makes the recursive embedding issue incredible unlikely.

It's certainly a probabilistic approach, but at some point the  
probability of collision because much less than the probability of  
disk corruption.

Antony Blakey
--------------------------
CTO, Linkuistics Pty Ltd
Ph: 0438 840 787

The truth does not change according to our ability to stomach it.
   -- Flannery O'Connor