couchdb-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Rhett Garber <rhe...@gmail.com>
Subject data loading
Date Wed, 04 Feb 2009 07:48:24 GMT
I'm playing with couchdb for the first time. I'm trying to load up
some data to play with but it seems REALLY slow.

I'm using python-couchdb as the client.

Are there any tips for getting data into couchdb in bulk ?
It seems entirely IO bound at the moment, but it isn't  clear to me why.

I'm loading a log file of about 1.2 gigs... processing it in pure
python is CPU bound and takes about 1.5 minutes to get through the
whole thing.

Loading in the couchdb, i've only got 30 megs in the last hour. That
30 megs has turned into 389 megs in the couchdb data file. That
doesn't seem like enough disk IO to cause this sort of delay.....
where is the time going ? network ?

Rhett

Mime
View raw message