incubator-couchdb-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Benjamin Young <benja...@couchone.com>
Subject Re: Sharing design documents between DBs
Date Tue, 02 Nov 2010 21:22:32 GMT
The one "catch" is that the majority of a CouchApp is related to querying
(map/reduce) and formatting (_show/_list) and validating (_update). While
the contents of the _design/app document may be the same, the disk storage
for the results of the map/reduce and _show/_list caching is specific (and
should be) to the specific database.

Managing which db the indexes are related to would get quite tedious
("context is expensive") and likely troublesome for CouchDB--hence the need
for sharing the _design/app document.

It is a different sort of pattern than the world's used to, but the
opportunities this model opens up (remote instances per client,
mobile/desktop installed versions of the app, etc) out way (at least IMO)
the cost of disk usage, etc.

It takes some getting used to, but it fits better than it feels at first
blush. :)

On Mon, Nov 1, 2010 at 4:56 PM, Gregor Frey <gkf@makkiato.de> wrote:

> Thanks for the very interesting discussion!
> Benjamin's proposal to distinguish between the replication of "app dbs" on
> different hosts and replicating the design documents between "customer dbs"
> on the same machine and CouchDB instance reduces the http-traffic and
> supports scaling over many machines.
> Nevertheless I wonder why, if I really want the design documents to be
> identical for all customers - and this is what multi-tenancy demands - I
> must replicate the app documents to every db on the same instance. The
> ability to replicate to different instances greatly provides for
> scalability. But wouldn't it be much simpler to operate and would consume
> less resources (don't forget the file-I/O!) if all the dbs on one instance
> could share the same application documents?
> Ciao
> Gregor
>
>
> On 01.11.2010 18:46, Benjamin Young wrote:
>
>> Martin,
>>
>> I'm not sure what your setup is, but if you had an "app" database (the
>> authoritative db for the _design doc you want to share with customers) on
>> the same instance of CouchDB with the customer's db's, then your
>> replication
>> requests would all be local--no IP connections what so ever.
>>
>> To do that in a multi-instance scenario, you could simply have the app
>> db's
>> continuously replicate between the instances, and locally replication to
>> the
>> customer db's on each particular instance. Or, if you don't want to use
>> replication between your instances, you could push your changes into the
>> "app" db's on each of the instances. In either case, you'd only be
>> generating HTTP traffic on the pushing of the app and/or replication of
>> the
>> "app" db's between the machines--so, very low network overhead in either
>> case.
>>
>> Later,
>> Benjamin
>>
>> On Mon, Nov 1, 2010 at 11:10 AM, Martin Higham<martin@ocastalabs.com
>> >wrote:
>>
>>
>>
>>> 'Continuous' replication to thousands of databases means thousands of
>>> permanent IP connections. The alternative is that you write a script that
>>> fires off replication requests in sequence for all your databases to
>>> perform
>>> the update. As you say, this shouldn't occur often.
>>>
>>> Martin
>>>
>>> On 1 November 2010 14:48, Benjamin Young<benjamin@couchone.com>  wrote:
>>>
>>>
>>>
>>>> Martin,
>>>>
>>>> Why not? It's only going to be sending changes. Unless you're constantly
>>>> updating your app installation or those changes are massive, you
>>>>
>>>>
>>> shouldn't
>>>
>>>
>>>> run into any trouble.
>>>>
>>>> Your other option is the "middle-ware" setup, but then you'd loose the
>>>> power
>>>> of application replication.
>>>>
>>>> Thoughts?
>>>>
>>>> Later,
>>>> Benjamin
>>>>
>>>> On Mon, Nov 1, 2010 at 10:30 AM, Martin Higham<martin@ocastalabs.com
>>>>
>>>>
>>>>> wrote:
>>>>>
>>>>>
>>>>
>>>>
>>>>> Until you have one DB per user and then you're looking at replicating
>>>>>
>>>>>
>>>> the
>>>
>>>
>>>> design doc to many thousands of databases and continuous replication
>>>>> doesn't
>>>>> make sense
>>>>>
>>>>> Martin
>>>>>
>>>>> On 1 November 2010 14:11, Benjamin Young<benjamin@couchone.com>
>>>>>
>>>>>
>>>> wrote:
>>>
>>>
>>>>
>>>>>
>>>>>> Hey Gregor,
>>>>>>
>>>>>> If you setup continuous replication between your various customer
>>>>>>
>>>>>>
>>>>> db's
>>>
>>>
>>>> and
>>>>>
>>>>>
>>>>>> your primary application database (which would likely only contain
>>>>>>
>>>>>>
>>>>> the
>>>
>>>
>>>> main
>>>>>
>>>>>
>>>>>> app's design doc), then publication of the app would automatically
be
>>>>>> "rolled out" to the various customer db's. Because these DB's would
>>>>>>
>>>>>>
>>>>> be
>>>
>>>
>>>>  "standalone" versions of the app, they could even be on multiple
>>>>>>
>>>>>>
>>>>> hosts
>>>
>>>
>>>>  running CouchDB, so you'd remove the single point of failure problem
>>>>>>
>>>>>>
>>>>> that
>>>>
>>>>
>>>>> most web-apps have--as they run (often) through a single server for
>>>>>>
>>>>>>
>>>>> all
>>>
>>>
>>>>  customers.
>>>>>>
>>>>>> Personally, that mode of "multi-tenant" app (via replication) is
>>>>>>
>>>>>>
>>>>> pretty
>>>
>>>
>>>>  exciting, and opens up new ways of dealing with load and application
>>>>>> distribution. Get's the mind reeling, or maybe that's the coffee
I
>>>>>>
>>>>>>
>>>>> just
>>>
>>>
>>>>  finished... :)
>>>>>>
>>>>>> Later, Gregor,
>>>>>> Benjamin
>>>>>>
>>>>>> On Mon, Nov 1, 2010 at 6:49 AM, Jan Lehnardt<jan@apache.org>
 wrote:
>>>>>>
>>>>>>
>>>>>>
>>>>>>> Hi Gregor,
>>>>>>>
>>>>>>> On 1 Nov 2010, at 08:12, Gregor Frey wrote:
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>> Hi,
>>>>>>>> when I followed the discussion about the setup of CouchDB
in a
>>>>>>>>
>>>>>>>>
>>>>>>> hosted
>>>>
>>>>
>>>>>  environment, I wondered whether it would be possible to share the
>>>>>>> application level software between multiple databases. This would
>>>>>>>
>>>>>>>
>>>>>> enable
>>>>>
>>>>>
>>>>>> a
>>>>>>
>>>>>>
>>>>>>> real multi-tenant set-up. Otherwise you must duplicate the
>>>>>>>
>>>>>>>
>>>>>> application
>>>>
>>>>
>>>>> with
>>>>>>
>>>>>>
>>>>>>> each new tenant.
>>>>>>>
>>>>>>>
>>>>>>>> Does anybody know whether and how CouchDB supports application
>>>>>>>>
>>>>>>>>
>>>>>>> sharing?
>>>>>
>>>>>
>>>>>> CouchDB does not support application or document sharing over
>>>>>>>
>>>>>>>
>>>>>> databases.
>>>>>
>>>>>
>>>>>> But nothing stops you from gradually replicating a new design doc
>>>>>>>
>>>>>>>
>>>>>> (the
>>>>
>>>>
>>>>>  application) to every database.
>>>>>>>
>>>>>>> Cheers
>>>>>>> Jan
>>>>>>> --
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>
>>>>
>>>
>>>
>>
>>
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message