incubator-couchdb-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Sam Bisbee <...@sbisbee.com>
Subject Re: Rapidly decreasing insert speed
Date Thu, 15 Mar 2012 18:38:30 GMT
On Thu, Mar 15, 2012 at 6:13 AM, Daniel Gonzalez <gonvaled@gonvaled.com> wrote:
> Hi,
>
> I am processing some input files and inserting the obtained records as
> CouchDB documents.
> I have noticed that the insert speed is decreasing in pace with
> database size increase.
>
> What I do is:
> 1) Read data from input file
> 2) Process the data to obtain the structured documents
> 3) Put the documents in a local buffer
> 4) As soon as the buffer has 1000 documents, perform a couchdb bulk insert
> 5) Repeat until input data has been fully processed

[snip]

Hi Daniel,

Would it be possible for you to post the test code you're running
someplace? Github, friendpaste, etc. It would help to see it run
locally and look at the code.

Cheers,

--
Sam Bisbee

Mime
View raw message