incubator-couchdb-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Chad Cross <chadcr...@gmail.com>
Subject Re: crawl webpages into couchdb
Date Wed, 09 Oct 2013 18:09:42 GMT
Affi,

CouchDB doesn't natively solve the web crawling issue.  I'm currently
experimenting with Scrapy (http://scrapy.org) for web crawling, but I
haven't advanced enough to start pushing my crawling data into CouchDB.
 Maybe some users out there have experience with Scrapy and CouchDB?

-Chad


On Wed, Oct 9, 2013 at 1:45 PM, Brad Rhoads <bdrhoa@gmail.com> wrote:

> Or better yet, casperjs.
> On Oct 7, 2013 3:37 PM, "Mark Hahn" <mark@reevuit.com> wrote:
>
> > Use node, phantomjs, and the nano couchdb driver.
> >
> >
> > On Mon, Oct 7, 2013 at 2:24 PM, affi <sw3et.poison@hotmail.com> wrote:
> >
> > > hi ,
> > > i am a beginner at couchdb and am learning it for a uni project. i have
> > > watched many tutorials on JSON and understand how to add documents.
> but i
> > > dont
> > > understand how to crawl webpages and store them in the couchdb
> database.
> > > would definitely appreciate some help with this. thanks
> > >
> > >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message