Mailing-List: contact couchdb-user-help@incubator.apache.org; run by ezmlm
Precedence: bulk
Reply-To: couchdb-user@incubator.apache.org
Received-SPF: neutral (athena.apache.org: local policy)
Message-ID: <a01fc5d10810281404j636e586dx56059f7a1f230b89@mail.gmail.com>
Date: Tue, 28 Oct 2008 21:04:54 +0000
From: "Nick Johnson" <arachnid@notdot.net>
To: couchdb-user@incubator.apache.org
Subject: Re: Getting only updates from a view
In-Reply-To: <7db9abd30810262054j3c9c9156se80be08449569dbf@mail.gmail.com>
MIME-Version: 1.0
Content-Type: multipart/alternative;
	boundary="----=_Part_40169_14404532.1225227894953"
References: <83FAD073-9DD0-495C-B70D-C587CA3563FD@msc-mobile.com>
	 <27d8d0930810260736g58acbd4bn61c417d3e1fd3b95@mail.gmail.com>
	 <E47B5491-BAF9-49ED-B736-C85D838DA1DC@mac.com>
	 <7db9abd30810262054j3c9c9156se80be08449569dbf@mail.gmail.com>

------=_Part_40169_14404532.1225227894953
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: 7bit
Content-Disposition: inline

On Mon, Oct 27, 2008 at 3:54 AM, kowsik <kowsik@gmail.com> wrote:

> The view server (couchjs) today is primarily a local process
> interacting with couch through stdin/stdout. This is the one that does
> all the map/reduce. I can envision at some point (not sure if and when
> it's planned), the view servers executing on remote machines through
> the exact same line-based JSON protocol (though using TCP) to heavily
> parallelize the map/reduce process. The incremental set of documents
> to be indexed can be easily chunked and distributed across this
> cluster. Until that's possible, the single machine solution is going
> to require some serious compute power (and lots of memory).


Hm. Actually, you could do this right now, by writing a view 'server' that
connects to an actual server over TCP (and a corresponding stub for the
other end). It would make a neat project, actually. The only real problem
would be convincing couch to spawn an appropriate number of threads for the
'real' parallelism available.


>
> IMHO, this [potential] parallelization (in addition to all the
> benefits of document-centric storage) is what makes couch incredibly
> attractive from a scaling perspective.
>
> K.
>
> On Sun, Oct 26, 2008 at 7:08 PM, Julien Guimont
> <julien.guimont@msc-mobile.com> wrote:
> > Thanks for the answer, pretty obvious.
> >
> > I would have 1000 to 10000 views to update periodically upon document
> > updates. There would be 5-10 updates a second and more than 500k
> documents.
> > Will couchdb scale in that case?
> >
> > Thank you,
> > Julien.
> >
> > On 26-Oct-08, at 10:36 AM, Ayende Rahien <ayende@ayende.com> wrote:
> >
> >> You create a view indexed by update date (or some other always
> >> incrementing
> >> value).Then you can ask to get the values by that value.
> >>
> >> On Sun, Oct 26, 2008 at 4:19 PM, Julien Guimont <
> >> julien.guimont@msc-mobile.com> wrote:
> >>
> >>> Hello,
> >>>
> >>> I am looking at CouchDB for a new project. So far it matches a lot of
> >>> requirements that would require ugly hacks using a traditional DB.
> >>>
> >>> One requirement I have is to always have the delta results of a view
> >>> (what
> >>> documents matching the view has changed since last update).
> >>>
> >>> I read http://wiki.apache.org/couchdb/RegeneratingViewsOnUpdate
> >>>
> >>> and I understood that the views are being reprocessed only with the
> >>> documents that have changed. (Am I wrong?)
> >>>
> >>> Well, if it is, can how can I get the view results only for those
> changed
> >>> documents?
> >>>
> >>> Thank you!
> >>> Julien.
> >>>
> >
>

------=_Part_40169_14404532.1225227894953--