Mailing-List: contact user-help@couchdb.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@couchdb.apache.org
Received-SPF: pass (nike.apache.org: domain of akat.metin@gmail.com designates
 209.85.219.211 as permitted sender)
DomainKey-Signature: a=rsa-sha1; c=nofws;
        d=gmail.com; s=gamma;
        h=mime-version:in-reply-to:references:date:message-id:subject:from:to
         :content-type;
        b=iMTVNRcrpo1h25pZ1TfG82W1zzRxweu2RCngYGNRKXn4chDo7mdELvdyuqppWHli21
         SWJeWlSDvrX3yOFc6u9V3ofGDFNORC2VFN+vgBWdMJA5MQx8Y7TgW4kUFIo+dh8ttQYb
         4nJJpOaOVcfoZeoj6zXurw1L7NeaEfrGtVgX4=
MIME-Version: 1.0
In-Reply-To: <4B8F971A.2080909@gmail.com>
References: <4B8EE0DE.6000205@gmail.com>
	 <3285B6DD-83C4-4F97-8CA8-DBDFFBC45B45@googlemail.com>
	 <4B8F971A.2080909@gmail.com>
Date: Thu, 4 Mar 2010 13:23:43 +0200
Message-ID: <98a246a11003040323m49223f22i6d6d2310ca04944f@mail.gmail.com>
Subject: Re: couchdb for genome data
From: Metin Akat <akat.metin@gmail.com>
To: user@couchdb.apache.org
Content-Type: text/plain; charset=ISO-8859-1

No, there is no limit.
And you can structure them in subdirectories by using a slash in the
database name.
Like "your/name/here"   will create 2 subdirectories with a database
called "here" at the bottom.

On Thu, Mar 4, 2010 at 1:18 PM, Tom Sante <tom.sante@gmail.com> wrote:
> Thanks. Are there any limits to the number of databases in couchdb? A few
> 1000 probably won't be a problem I guess?
>
> On 4/03/10 11:49, Simon Metson wrote:
>>
>> Hi,
>> Why not use a database per experiment? Do you need to process data
>> across experiments? Can you store your raw data in individual databases
>> and then pull summary data into a single database?
>> Cheers
>> Simon
>>
>> On 3 Mar 2010, at 22:21, Tom Sante wrote:
>>
>>> Hi
>>>
>>> The data is now stored in a mysql table with about a billion (1000
>>> million) rows.
>>> These rows are the data of a genetic test (arrayCGH) and build up like
>>> this:
>>>
>>> Every experiment (a few thousand of them total) contains measurements
>>> of about 180000 genetic probes. This raw data will be analyzed and the
>>> values run through different algorithms, so every probe needs to store
>>> more than 1 value after the analysis is done. The values of different
>>> analysis are now stored in columns in that table making it a pain if
>>> we have to add a analysis to the table not yet part of the existing
>>> columns. This is why a schema free document based DB is probably a
>>> better fit.
>>> The initial idea was to give each probe a separate document, and when
>>> the original value is transform to an other value store this in the
>>> same document.
>>>
>>> {
>>> "probe_id" : 1234567890,
>>> "experiment_id" : 1234567890,
>>> "raw_value" : 0.43524,
>>> "analysis": { "cbs" : 0.436, "CBS+GLAD" : 0.4356 }
>>> }
>>>
>>> Once added to the database almost all changes to the data will be
>>> contained within an experiment.
>>>
>>> MongoDB has something like collections that would be a appropriate
>>> abstraction ~ experiment. But in couchdb I would have to add all these
>>> probe documents in 1 big database without collections. So if I only
>>> make changes to probes within an experiment this would influence the
>>> views of all the other billions document in the db. Because of the
>>> large number of documents it would be good to know beforehand what the
>>> implications are of this performance wise?
>>>
>>> Any suggestions are welcome.
>>>
>>> Tom
>>
>
>