couchdb-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Rhett Garber <rhe...@gmail.com>
Subject Re: data loading
Date Tue, 17 Feb 2009 09:21:19 GMT
Finally put a writeup of some of my findings up:
http://nullhole.com/2009/02/17/first-impressions-of-couchdb/

Any comments / further advice very welcome.

Rhett

On Tue, Feb 3, 2009 at 11:48 PM, Rhett Garber <rhettg@gmail.com> wrote:
> I'm playing with couchdb for the first time. I'm trying to load up
> some data to play with but it seems REALLY slow.
>
> I'm using python-couchdb as the client.
>
> Are there any tips for getting data into couchdb in bulk ?
> It seems entirely IO bound at the moment, but it isn't  clear to me why.
>
> I'm loading a log file of about 1.2 gigs... processing it in pure
> python is CPU bound and takes about 1.5 minutes to get through the
> whole thing.
>
> Loading in the couchdb, i've only got 30 megs in the last hour. That
> 30 megs has turned into 389 megs in the couchdb data file. That
> doesn't seem like enough disk IO to cause this sort of delay.....
> where is the time going ? network ?
>
> Rhett
>

Mime
View raw message