On Dec 31, 2008, at 7:40 PM, Antony Blakey wrote:

>
> On 31/12/2008, at 11:29 PM, Geir Magnusson Jr. wrote:
>
>> What trouble?  I think this is *exactly* what should be done - have  
>> CouchDB store documents that are :
>>
>> {
>>   metadata : { _rev : X, _id : Y, _woogie: Z, .... anything that  
>> needs to be added in the future, like other metadata like last  
>> update date... },
>>   userdata : {  .... the document you want to store .... }
>> }
>>
>> and then offer APIs that let you :
>>
>> a) get to this document, for libraries and clients that know they  
>> are talking to Couch and want to manipulate at this level
>>
>> b) return and accept the userdocument directly, for clients that  
>> just want to consume or produce  JSON data, w/o caring about the  
>> internal housekeeping
>
> One of the issues complicating the logic of this discussion is that  
> the document id is both metadata and, conceptually, a document member.

Well, I don't understand why it has to be. Certainly it's a  
convenience, and I wonder how much of current thinking has been  
influenced by the fact that this what people are used to.

I can understand why CDB needs a unique document identifier, and it  
certainly would be nice to have the option of having it shoved into  
the user doc on creation.  But

a) I think that I should have the choice as to what that identifier  
is  (e.g.  Configure the database to inject the couch metadata _id as  
"_couchID" or whatever...)

b) I should have the choice to not have it injected at all

So why do I think this is a problem?  The 10gen appserver auto-injects  
an id field into the JSON documents that are stored in our database,  
Mongo.  Can you guess what the key is?  Yep - "_id"

So how can I roundtrip a doc from 10gen through couch and back?  I  
can't.

I've made the same argument at 10gen - that I should be able to set  
the identifier (and that it shouldn't be in the doc in the first place).

Then, I'd just have a doc with

{
    _couchID : ....
    _mongoID : ....
     ... data...
}

(if I chose to shove the ID into the doc)


> That's why, although the purest model is to have the userdata as a  
> member within a Couch document as you suggest, this doesn't look  
> that appealing:
>
> {
>  metadata: {
>    id: ...
>    rev: ...
>    ...
>  }
>  data: {
>    ... the user's document ...
>  }
> }

I can see how this isn't appealing from the perspective of current  
API's, but a rethinking of this issue (_id and _rev) also warrants a  
re-thinking of the APIs to deal with this.

E.g. an API that lets me get a) the whole doc above  b) metadata only  
c) userdata only

>
>
> Furthermore, from a scalability perspective, always having the  
> metadata when you have the document, isn't a problem - the metadata  
> is constrained.

And from what I understand, it already exists in that manner, right?   
I mean, for efficiency, I'd guess that the _id, _rev and in the  
future, other metadata (like insert date, last modificationdate...)  
would be kept outside of the doc, so that they can be read and updated  
w/o having to serialized/deserialize the whole user document.

> The reverse situation of always having the data when you have the  
> metadata, is not constrained because the data is arbitrarily large.  
> IMO this means that a solution such as this:
>
> {
>  id: ...
>  rev: ...
>  ...
>  data: {
>    ... the user's document ...
>  }
> }
>
> isn't such a good idea compared to this:
>
> {
>  _metadata: {
>    id: ...
>    rev: ...
>  }
>  ... the user's document ...
> }

That only solves the problem in that there's only one reserved magical  
key (_metadata), but I don't think that really changes anything.  You  
still need to make sure any document you want to store in couch  
doesn't have a top-level _metadata element.

And while I don't know how couch works internally, we *are* really  
only talking about how the data is returned on an API call via the  
REST API or what I assume is an internal API for the M/R View stuff.

If you had an API that let you choose all, metaonly or useronly, you  
could not be burdened with stuff you didn't want or need.

>
> Unfortunately the reserved token makes the structure non-reflexive  
> without transformation, and although that's not currently an issue,  
> I can imagine it complicating certain use-cases. It makes the system  
> more complicated to reason about.
>
> I'm struggling to objectively evaluate this model and your reflexive  
> model - given Damien's attitude to this issue, my motivation to do  
> so is somewhat depressed :/

If you could point me to an explanation of why changing this is bad,  
I'd love to catch up on the discussion.  I assume it's a technical  
reason?

geir

>
>
> Antony Blakey
> -------------
> CTO, Linkuistics Pty Ltd
> Ph: 0438 840 787
>
> Did you hear about the Buddhist who refused Novocain during a root  
> canal?
> His goal: transcend dental medication.
>
>