Mailing-List: contact user-help@couchdb.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@couchdb.apache.org
Received-SPF: pass (nike.apache.org: local policy)
From: Jim Klo <jim.klo@sri.com>
To: "<user@couchdb.apache.org>" <user@couchdb.apache.org>
Subject: Re: Cryptograhically signed docs...
Thread-Topic: Cryptograhically signed docs...
Thread-Index: AQHNWTP6QRnFgf7rFUiFg2X9Tdl1iZcYPjmAgACHjoCAApS7AA==
Date: Thu, 5 Jul 2012 16:31:42 +0000
Message-ID: <7149BF67-2B75-4687-868C-47F8811ED26A@sri.com>
References: 
 <CAMc++1U53sht7v24j+ktQyNe=e2cDTkCsdsX3Bm-KY-0rLR=Eg@mail.gmail.com>
 <79061073-7E58-4CBC-9DC1-8A98C6811796@sri.com>
 <0848BFC4-1340-4FF9-A04D-EE6D205A90A0@couchbase.com>
In-Reply-To: <0848BFC4-1340-4FF9-A04D-EE6D205A90A0@couchbase.com>
Accept-Language: en-US
Content-Language: en-US
Content-Type: multipart/alternative;
	boundary="_000_7149BF672B754687868C47F8811ED26Asricom_"
MIME-Version: 1.0

--_000_7149BF672B754687868C47F8811ED26Asricom_
Content-Type: text/plain; charset="Windows-1252"
Content-Transfer-Encoding: quoted-printable

Hi Jens,

Agreed - however our use case wouldn't work well with solution you propose.=
  Our documents are signed before they are inserted into CouchDB http://goo=
.gl/vlJwO (and actually before certain metadata fields are attached to the =
object).  That's one area where I think your proposal is lacking is that th=
ere's not really a method to exclude fields from the signature except via u=
nderscore fields. The key thing to note is we had a document model before o=
bject signing requirement; so we had to design a signature solution that wa=
s additive in order to not change object model field names for risk of blow=
ing backwards compatibility.  If there was way with your solution, similar =
to the security object, store an model to XOR against to compute/validate t=
he signature that might be fine.

On Jul 3, 2012, at 6:07 PM, Jens Alfke wrote:


On Jul 3, 2012, at 10:01 AM, Jim Klo wrote:

Yes, and as a matter of fact, i just got digital signature validation using=
 OpenPGP within a map function working a few minutes ago!
Here's a link to the relevant code: https://github.com/jimklo/TheCollector/=
blob/master/dataservices/thecollector-resources/views/lib/sig_utils.js

As far as I can tell, this code uses a data schema where the signed content=
s are wrapped in some kind of OpenPGP encoding:

       var msg_list =3D openpgp.read_message(doc.digital_signature.signatur=
e);
       for (var i=3D0; i<msg_list.length; i++) {
           isValid |=3D msg_list[i].verifySignature();
       }

It looks like msg_list is the actual document payload, which has to be deco=
ded using openpgp.read_message.


Correct - and yes agreed not exactly the most efficient.  Possibly using a =
detached signature would be better, however because we also identified that=
 a canonical JSON serialization has not been widely defined or accepted we =
went for a translation of selected signed fields to bencoding then using Op=
enPGP, clearsign a SHA256 hash of the bencoded JSON.

This is IMHO not a very good solution because it hides the document content=
s away =97 for example, all the map functions and any app logic that uses d=
ocuments will have to know to call read_message, which will also make them =
slower.

Presumably the performance hit is only a once per view hit, which I can liv=
e with in our UC. I'm not sure I'm following though - we don't hide the doc=
ument contents at all, as we use clearsign, unless you were assuming we wer=
e encrypting too.  I'd agree that I don't necessarily like that each map mu=
st apply the check; however for our UC, that's acceptable, since not all do=
cuments are signed.  Additionally, moving this code from a map function int=
o a validate document update function would solve the need of having to val=
idate within each map function; potentially marking each document with a _i=
svalidsignature field upon insert/update.


The schema I implemented (see my previous message) doesn't alter the basic =
document format. The signature is in a nested object but applies to the ent=
ire document contents (minus the signature itself of course). There's no ne=
ed to change any code that reads documents; the only time you have to know =
about the signature scheme is while verifying the signature. It's even poss=
ible to have multiple signatures on a document.


Essentially we've both aligned to more or less the same solution, and ident=
ified similar challenges. The one issue both of us face is the encoding of =
floating point values. Our rule is to convert FPV to a base 10 string repre=
sentation, before bencoding. Otherwise things like ratings or other numeric=
ally scaled values get lost in the serialization process and non-unique has=
hes become pointless to sign.

If you haven't been following IETF's JOSE, you should take a look at it htt=
p://datatracker.ietf.org/doc/draft-ietf-jose-json-web-signature/?include_te=
xt=3D1  It's not very CouchDB friendly, however I wonder if there would be =
a way to make couch use it; Say you can insert and update docs using a JOSE=
 payload instead of a JSON payload, CouchDB could deserialize the b64 encod=
ing back into JSON for users who don't care to validate signatures, but the=
n optionally could GET the documents back as a list of JOSE payloads..  The=
 nice thing about their solution is there's no need for a canonical represe=
ntation, the bad thing is there's no way to deal with 'unsigned fields'.

I'm certainly open to some collaboration here, as I'm not too happy with be=
ncoding and FWIW canonical JSON is very difficult to make portable (ordered=
 maps aren't supported across many languages) I've been considering replaci=
ng bencoding with Tagged Netstrings http://tnetstrings.org/ which provides =
a good translation that is more portable and include floating point, and bo=
olean values.

=97Jens

Jim Klo
Senior Software Engineer
Center for Software Engineering
SRI International

--_000_7149BF672B754687868C47F8811ED26Asricom_--