incubator-couchdb-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Mark Deibert <mark.deib...@gmail.com>
Subject Re: crawl webpages into couchdb
Date Wed, 09 Oct 2013 18:25:29 GMT
Are we in a CouchDB group or a "web crawler" apps group? :-/


On Wed, Oct 9, 2013 at 2:09 PM, Chad Cross <chadcross@gmail.com> wrote:

> Affi,
>
> CouchDB doesn't natively solve the web crawling issue.  I'm currently
> experimenting with Scrapy (http://scrapy.org) for web crawling, but I
> haven't advanced enough to start pushing my crawling data into CouchDB.
>  Maybe some users out there have experience with Scrapy and CouchDB?
>
> -Chad
>
>
> On Wed, Oct 9, 2013 at 1:45 PM, Brad Rhoads <bdrhoa@gmail.com> wrote:
>
> > Or better yet, casperjs.
> > On Oct 7, 2013 3:37 PM, "Mark Hahn" <mark@reevuit.com> wrote:
> >
> > > Use node, phantomjs, and the nano couchdb driver.
> > >
> > >
> > > On Mon, Oct 7, 2013 at 2:24 PM, affi <sw3et.poison@hotmail.com> wrote:
> > >
> > > > hi ,
> > > > i am a beginner at couchdb and am learning it for a uni project. i
> have
> > > > watched many tutorials on JSON and understand how to add documents.
> > but i
> > > > dont
> > > > understand how to crawl webpages and store them in the couchdb
> > database.
> > > > would definitely appreciate some help with this. thanks
> > > >
> > > >
> > >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message