Mailing-List: contact user-help@couchdb.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@couchdb.apache.org
Received-SPF: neutral (nike.apache.org: local policy)
Message-ID: 
 <3c42307e64dc9ecee1b595858e3b81fe.squirrel@secure61.inmotionhosting.com>
In-Reply-To: <7CC02CFB-C515-478C-813C-274859EE5D4B@couchbase.com>
References: 
    <CAHKsR692SAqfg3aPvOJyp+HftDQtA5Vf8PNLG+OVk2p9z_9RDw@mail.gmail.com>
    <b24ecbb5fe9bb8bcd5beaf3d5ecc251f.squirrel@secure61.inmotionhosting.com>
    <CABvT1DFaw6RwK4ZX8WVk109o3Tt_5XWCNJ9bydHR=_d0b6CrPw@mail.gmail.com>
    <7caa2578e04507ec25f97df01e5d66ab.squirrel@secure61.inmotionhosting.com>
    <FE5952D2-D2ED-4D21-BB26-4E6206317931@couchbase.com>
    <2ba5948b99a136aebad13e1cfcd8b525.squirrel@secure61.inmotionhosting.com>
    <7CC02CFB-C515-478C-813C-274859EE5D4B@couchbase.com>
Date: Wed, 9 May 2012 14:53:24 -0700
Subject: Re: include_docs with _all_docs
From: dan@danmyersconsulting.com
To: user@couchdb.apache.org
User-Agent: SquirrelMail/1.4.21
MIME-Version: 1.0
Content-Type: text/plain;charset=iso-8859-1
Content-Transfer-Encoding: 8bit
Importance: Normal

Jens,

1. I'm in data management. There is a strong business case for having
robust business descriptions for each column (where it came from, what it
means, who entered it, how it's calculated...etc). Risk officers need to
analyze what data is in a db that (aka after a developer builds the
database and moves on nobody will know what attributes exist--without
reverse engineering the application). Business analysts have no way of
conducting Business Intelligence without knowing what they are querying.
How do you do this without descriptions?

2. Great this is helpful, I'll research more.

3. Trying to accomplish what is discussed in #1, but at the "column"
level. How would I count (or any other business analytic) tweets from
people in US, Japan, Brazil if my docs are tweets by IP, Country, Date?
How do I know that all tweets have "Country" on them if there isn't any
metadata to say so? If the follow-up business question is what the counts
are by country by day, how do I form a query by "Date" if I don't know
that the "column" for date even exists without metadata?

> On May 9, 2012, at 1:21 PM, <dan@danmyersconsulting.com>
> <dan@danmyersconsulting.com> wrote:
>
>> 1. If I don't know what attributes exist in the database there is no way
>> to write a query about those attributes on documents.
>
> Usually you know something about the schema of the data in a database
> before writing code to operate on it. CouchDB doesn�t enforce any schema,
> but you�d generally define what properties you put in documents
> beforehand.
>
> What kind of program are you writing? I�m not an RDBMS guru but I�ve
> written some apps that use SQL databases, and I haven�t had to introspect
> the metadata of the table columns. I�ve defined my tables and then written
> my queries using the columns I defined.
>
> Keep in mind that, to some degree, it doesn�t make sense to ask �what
> attributes exist in the database� because every document could have a
> different set of attributes. It�s very unlikely but possible. CouchDB has
> nothing comparable to �columns� in SQL; every document is an independent
> bag of JSON.
>
>> 2. If I create my own attribute of a document which stores metadata for
>> that document it works but if that metadata attribute isn't a required
>> attribute for all docs then how can I enforce the collection of metadata
>> which supports #1 above?
>
> You can add a validation function to a design document that will enforce
> the presence of that attribute, or any structure you want your documents
> to have.
>
>> 3. I can create a attribute that would provide a description of a
>> document, e.g.#2 above (aka metadata) but how do I create an attribute
>> of
>> a document attribute? (e.g. an description for a document attribute
>> called
>> "telephone", which is now an attribute of an attribute or
>> meta-attribute)?
>
> I really have no idea where you�re going with this. It�s awfully abstract.
> What are you trying to accomplish?
>
> �Jens