incubator-couchdb-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Mehdi El Fadil <mehdi.elfa...@mango-is.com>
Subject Re: data integration to couchdb
Date Tue, 21 Jun 2011 12:57:08 GMT
Thanks for the answer Sean,

but I am not sure whether this is what I need. From my understanding, the
_changes API only listens to what happens inside couchdb. What I need to do
is to listen to the outside world (via websites APIs, web services, RSS...)
at server side, and then only insert or update my data in couchdb.

Am I getting the _changes API wrong, would it allow to do so? I feel like I
would need another server side technology such as node.js.

Mehdi.


On Tue, Jun 21, 2011 at 2:18 PM, Sean Copenhaver
<sean.copenhaver@gmail.com>wrote:

> I could not comment on the performance but you can use the _changes API to
> get a soft real-time stream of changes to the database. It has a few
> options
> including listening continuously and applying filters to only receive what
> you are interested in:
>
> http://guide.couchdb.org/draft/notifications.html
>
> On Tue, Jun 21, 2011 at 7:15 AM, Mehdi El Fadil
> <mehdi.elfadil@mango-is.com>wrote:
>
> > Hello,
> >
> > I have to integrate data into couchdb. The following situations will be
> > possible:
> >
> >   - extract data in real time (or near real-time) from a somewhere
> online,
> >   process it and insert in couchdb
> >   - run batches to get data from some location (eg flat file available
> >   online), process data and insert / update documents inside couchdb
> >
> > My initial plan was to use couchdb both for storage and as application
> > server, to keep a simple architecture. However, my data integration
> > requirement might not be feasible easily using only views, list, and show
> > functions.
> >
> >   1. How would I listen to real-time messages from the outside world?
> >   2. Would there be any way to have reasonable performance in the data
> >   processing (filtering, aggregation) if done from couchdb?
> >
> > Thanks in advance for your advice.
> >
> > cheers,
> >
> > mehdi
> >
>
>
>
> --
> “The limits of language are the limits of one's world. “ -Ludwig von
> Wittgenstein
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message