incubator-couchdb-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Paul Hirst <paul.hi...@sophos.com>
Subject Re: multiple database or many doc types in one database?
Date Mon, 21 Mar 2011 09:37:01 GMT
On Thu, 2011-03-17 at 14:48 +0000, Kevin R. Coombes wrote:
>
> On 3/17/2011 3:46 AM, Paul Hirst wrote:
> > On Thu, 2011-03-17 at 04:51 +0000, Dominic Tarr wrote:
> >> is it a good idea to
> >>
> >>    1. separate documents for different types of thing into different
> >> databases?
> >>
> >> or is it better to
> >>
> >>    2. just have a type='whatever' and check for that in your views?
> > I am using option 2. I prepend type_ to my ids to avoid keys clashing
> > and also have a type field in the body which I use in my views.
> >
> > However I'm really regretting doing it this way as I only have two
> > documents types. I have 0.25 million of type A and 28 million of type B.
> > I often want to change the views regarding type A documents and have to
> > wait about 12 hours for it to rebuild. If I had separate databases this
> > would be far less of a problem as the type A views would rebuild in a
> > few minutes.
> What happens if you simply seprate the views into two different design
> documents?  If the views for type A were in one design document, then
> you would at least avoid rebuilding the views that work with type B
> (which at least suggest that the only time hit would come from having to
> execute "if (type == 'A') emit(stuff)" on the type B documents.

Actually it's already separate design documents. But the cost of loading
~27.75 million documents from the disk, json encoding and sending to the
view server is still very significant, even if the view server only does
a simple 'if' test and doesn't return anything.

This led me to think it would be great if documents could be tagged
somehow and then views could choose which documents to include based on
metadata tags. Then I figured this is almost exactly what I would get
using different databases.



Sophos Limited, The Pentagon, Abingdon Science Park, Abingdon, OX14 3YP, United Kingdom.
Company Reg No 2096520. VAT Reg No GB 991 2418 08.

Mime
View raw message