couchdb-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Robert Newson <rnew...@apache.org>
Subject Re: database design question: concurrent writes
Date Thu, 13 Dec 2012 16:46:30 GMT
Views are also sharded.

It's common for a node to host multiple shards of the same database,
so we already have this 'concurrent writes' notion, if I've
interpreted it correctly.

On 13 December 2012 16:36, Hans J Schroeder <hs@cloudno.de> wrote:
>
> On Dec 13, 2012, at 5:06 PM, Benoit Chesneau <bchesneau@gmail.com> wrote:
>
>> Hi all,
>>
>>
>> This morning I was back reading a lot of fundamentals about  databases and
>> such and was asking myself how we could increase the number of concurrent
>> writes.
>>
>> These days the theory is that it will be solved by sharding the databases
>> in multiples database files and merging results of the queries. Since the
>> databases will be shareded then the writes on the same db will be
>> concurrents. A map of the shards willl be kept aside. All of this thanks to
>> the introduction of bigcouch.
>>
>> The question I have is why don't we already do that? Ie balancing datas on
>> different files on one db? for example the db folder could be
>>
>> database/XY.couch
>>
>> where XY are the first letters of an id or content hash or any consistent
>> hashing method.
>>
>> I am currently asking myself such question because I am wondering how will
>> the backup works when couchdb will be used as a single node. How to backup
>> only one db without having to query for the mapping and such? How to keep
>> it it simple.
>>
>> Related to that why did bigcouch used that design? Why mapping shards in a
>> db database instead of having some kind of natural balancing on the fs and
>> having a consistent hashing algorithm used to balance on different
>> machines/vms as well ?
>>
>>
>> - benoƮt
>
>
> Hi,
>
> That's like "horizontal partitioning" in conventional databases and I think its a great
idea. Having a writer process for each partition will make it scale.
>
> Does Bigcouch have anything for the view files too or are they just sharding the backing
files?
>
> - Hans

Mime
View raw message