Mailing-List: contact user-help@couchdb.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@couchdb.apache.org
Received-SPF: pass (nike.apache.org: domain of chadcross@gmail.com designates
 74.125.82.49 as permitted sender)
MIME-Version: 1.0
In-Reply-To: 
 <CABu8_fUqpB+FKJgo95JLZ-e2kh1LY2ji-r-Mb7bBoz6zAmGNOQ@mail.gmail.com>
References: <loom.20131007T232120-406@post.gmane.org>
	<CAK0S_BAeNq1z6+QEig3D=MJ1kVOAmZUqTOSRwp3Cd=Ad4RoCWQ@mail.gmail.com>
	<CABu8_fUqpB+FKJgo95JLZ-e2kh1LY2ji-r-Mb7bBoz6zAmGNOQ@mail.gmail.com>
Date: Wed, 9 Oct 2013 14:09:42 -0400
Message-ID: 
 <CADzYs-NAVocdGGhS-2cOrV3iG7xBrQwqWtkDb0zMs-CVXkQBbA@mail.gmail.com>
Subject: Re: crawl webpages into couchdb
From: Chad Cross <chadcross@gmail.com>
To: user@couchdb.apache.org
Content-Type: multipart/alternative; boundary=001a11c25d3213a4b304e852cad8

--001a11c25d3213a4b304e852cad8
Content-Type: text/plain; charset=ISO-8859-1

Affi,

CouchDB doesn't natively solve the web crawling issue.  I'm currently
experimenting with Scrapy (http://scrapy.org) for web crawling, but I
haven't advanced enough to start pushing my crawling data into CouchDB.
 Maybe some users out there have experience with Scrapy and CouchDB?

-Chad


On Wed, Oct 9, 2013 at 1:45 PM, Brad Rhoads <bdrhoa@gmail.com> wrote:

> Or better yet, casperjs.
> On Oct 7, 2013 3:37 PM, "Mark Hahn" <mark@reevuit.com> wrote:
>
> > Use node, phantomjs, and the nano couchdb driver.
> >
> >
> > On Mon, Oct 7, 2013 at 2:24 PM, affi <sw3et.poison@hotmail.com> wrote:
> >
> > > hi ,
> > > i am a beginner at couchdb and am learning it for a uni project. i have
> > > watched many tutorials on JSON and understand how to add documents.
> but i
> > > dont
> > > understand how to crawl webpages and store them in the couchdb
> database.
> > > would definitely appreciate some help with this. thanks
> > >
> > >
> >
>

--001a11c25d3213a4b304e852cad8--