Mailing-List: contact couchdb-user-help@incubator.apache.org; run by ezmlm
Precedence: bulk
Reply-To: couchdb-user@incubator.apache.org
Received-SPF: pass (athena.apache.org: domain of kowsik@gmail.com designates
 74.125.92.147 as permitted sender)
DomainKey-Signature: a=rsa-sha1; c=nofws;
        d=gmail.com; s=gamma;
        h=message-id:date:from:to:subject:in-reply-to:mime-version
         :content-type:content-transfer-encoding:content-disposition
         :references;
        b=dt68RmADONEOZOvWTo/1khqZI4nzZzRmTFGE6sEObH7QF4kvc/9S8qQsWei0HjpwZO
         hUJkcGjOl+LTrBJRo2wWIBRrKzh9Gt0HkZuqdqwml+zWWh+fghkP4G1znFt3YGR5GUgq
         R7zfzvTOssELgVy4BHKq7M7OInvhvMsRIJYto=
Message-ID: <7db9abd30810262054j3c9c9156se80be08449569dbf@mail.gmail.com>
Date: Sun, 26 Oct 2008 20:54:15 -0700
From: kowsik <kowsik@gmail.com>
To: couchdb-user@incubator.apache.org
Subject: Re: Getting only updates from a view
In-Reply-To: <E47B5491-BAF9-49ED-B736-C85D838DA1DC@mac.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit
Content-Disposition: inline
References: <83FAD073-9DD0-495C-B70D-C587CA3563FD@msc-mobile.com>
	 <27d8d0930810260736g58acbd4bn61c417d3e1fd3b95@mail.gmail.com>
	 <E47B5491-BAF9-49ED-B736-C85D838DA1DC@mac.com>

The view server (couchjs) today is primarily a local process
interacting with couch through stdin/stdout. This is the one that does
all the map/reduce. I can envision at some point (not sure if and when
it's planned), the view servers executing on remote machines through
the exact same line-based JSON protocol (though using TCP) to heavily
parallelize the map/reduce process. The incremental set of documents
to be indexed can be easily chunked and distributed across this
cluster. Until that's possible, the single machine solution is going
to require some serious compute power (and lots of memory).

IMHO, this [potential] parallelization (in addition to all the
benefits of document-centric storage) is what makes couch incredibly
attractive from a scaling perspective.

K.

On Sun, Oct 26, 2008 at 7:08 PM, Julien Guimont
<julien.guimont@msc-mobile.com> wrote:
> Thanks for the answer, pretty obvious.
>
> I would have 1000 to 10000 views to update periodically upon document
> updates. There would be 5-10 updates a second and more than 500k documents.
> Will couchdb scale in that case?
>
> Thank you,
> Julien.
>
> On 26-Oct-08, at 10:36 AM, Ayende Rahien <ayende@ayende.com> wrote:
>
>> You create a view indexed by update date (or some other always
>> incrementing
>> value).Then you can ask to get the values by that value.
>>
>> On Sun, Oct 26, 2008 at 4:19 PM, Julien Guimont <
>> julien.guimont@msc-mobile.com> wrote:
>>
>>> Hello,
>>>
>>> I am looking at CouchDB for a new project. So far it matches a lot of
>>> requirements that would require ugly hacks using a traditional DB.
>>>
>>> One requirement I have is to always have the delta results of a view
>>> (what
>>> documents matching the view has changed since last update).
>>>
>>> I read http://wiki.apache.org/couchdb/RegeneratingViewsOnUpdate
>>>
>>> and I understood that the views are being reprocessed only with the
>>> documents that have changed. (Am I wrong?)
>>>
>>> Well, if it is, can how can I get the view results only for those changed
>>> documents?
>>>
>>> Thank you!
>>> Julien.
>>>
>