Mailing-List: contact dev-help@couchdb.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@couchdb.apache.org
Received-SPF: neutral (athena.apache.org: local policy)
Content-Type: text/plain; charset=iso-8859-1
Mime-Version: 1.0 (Mac OS X Mail 6.2 \(1499\))
Subject: Re: database design question: concurrent writes
From: Hans J Schroeder <hs@cloudno.de>
In-Reply-To: 
 <CAJNb-9rS6PMPSza==0kELeSPrs36ht0OVga4O4sq=bWftjwiCA@mail.gmail.com>
Date: Thu, 13 Dec 2012 17:36:00 +0100
Content-Transfer-Encoding: quoted-printable
Message-Id: <C2440791-D5C3-4F15-AAE3-8C33F92D2759@cloudno.de>
References: 
 <CAJNb-9rS6PMPSza==0kELeSPrs36ht0OVga4O4sq=bWftjwiCA@mail.gmail.com>
To: dev@couchdb.apache.org


On Dec 13, 2012, at 5:06 PM, Benoit Chesneau <bchesneau@gmail.com> =
wrote:

> Hi all,
>=20
>=20
> This morning I was back reading a lot of fundamentals about  databases =
and
> such and was asking myself how we could increase the number of =
concurrent
> writes.
>=20
> These days the theory is that it will be solved by sharding the =
databases
> in multiples database files and merging results of the queries. Since =
the
> databases will be shareded then the writes on the same db will be
> concurrents. A map of the shards willl be kept aside. All of this =
thanks to
> the introduction of bigcouch.
>=20
> The question I have is why don't we already do that? Ie balancing =
datas on
> different files on one db? for example the db folder could be
>=20
> database/XY.couch
>=20
> where XY are the first letters of an id or content hash or any =
consistent
> hashing method.
>=20
> I am currently asking myself such question because I am wondering how =
will
> the backup works when couchdb will be used as a single node. How to =
backup
> only one db without having to query for the mapping and such? How to =
keep
> it it simple.
>=20
> Related to that why did bigcouch used that design? Why mapping shards =
in a
> db database instead of having some kind of natural balancing on the fs =
and
> having a consistent hashing algorithm used to balance on different
> machines/vms as well ?
>=20
>=20
> - beno=EEt


Hi,=20

That's like "horizontal partitioning" in conventional databases and I =
think its a great idea. Having a writer process for each partition will =
make it scale.

Does Bigcouch have anything for the view files too or are they just =
sharding the backing files?

- Hans=