incubator-couchdb-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Nick Johnson" <arach...@notdot.net>
Subject Re: Getting only updates from a view
Date Tue, 28 Oct 2008 21:04:54 GMT
On Mon, Oct 27, 2008 at 3:54 AM, kowsik <kowsik@gmail.com> wrote:

> The view server (couchjs) today is primarily a local process
> interacting with couch through stdin/stdout. This is the one that does
> all the map/reduce. I can envision at some point (not sure if and when
> it's planned), the view servers executing on remote machines through
> the exact same line-based JSON protocol (though using TCP) to heavily
> parallelize the map/reduce process. The incremental set of documents
> to be indexed can be easily chunked and distributed across this
> cluster. Until that's possible, the single machine solution is going
> to require some serious compute power (and lots of memory).


Hm. Actually, you could do this right now, by writing a view 'server' that
connects to an actual server over TCP (and a corresponding stub for the
other end). It would make a neat project, actually. The only real problem
would be convincing couch to spawn an appropriate number of threads for the
'real' parallelism available.


>
> IMHO, this [potential] parallelization (in addition to all the
> benefits of document-centric storage) is what makes couch incredibly
> attractive from a scaling perspective.
>
> K.
>
> On Sun, Oct 26, 2008 at 7:08 PM, Julien Guimont
> <julien.guimont@msc-mobile.com> wrote:
> > Thanks for the answer, pretty obvious.
> >
> > I would have 1000 to 10000 views to update periodically upon document
> > updates. There would be 5-10 updates a second and more than 500k
> documents.
> > Will couchdb scale in that case?
> >
> > Thank you,
> > Julien.
> >
> > On 26-Oct-08, at 10:36 AM, Ayende Rahien <ayende@ayende.com> wrote:
> >
> >> You create a view indexed by update date (or some other always
> >> incrementing
> >> value).Then you can ask to get the values by that value.
> >>
> >> On Sun, Oct 26, 2008 at 4:19 PM, Julien Guimont <
> >> julien.guimont@msc-mobile.com> wrote:
> >>
> >>> Hello,
> >>>
> >>> I am looking at CouchDB for a new project. So far it matches a lot of
> >>> requirements that would require ugly hacks using a traditional DB.
> >>>
> >>> One requirement I have is to always have the delta results of a view
> >>> (what
> >>> documents matching the view has changed since last update).
> >>>
> >>> I read http://wiki.apache.org/couchdb/RegeneratingViewsOnUpdate
> >>>
> >>> and I understood that the views are being reprocessed only with the
> >>> documents that have changed. (Am I wrong?)
> >>>
> >>> Well, if it is, can how can I get the view results only for those
> changed
> >>> documents?
> >>>
> >>> Thank you!
> >>> Julien.
> >>>
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message