incubator-couchdb-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Calum Miller <>
Subject Re: Bulk Load
Date Sun, 14 Sep 2008 00:14:32 GMT
Hi Chris,

Many thanks for your prompt response.

Storing  a complete new version of each bond/instrument every day seems 
a tad excessive. You can imagine how fast the database will grow 
overtime if a unique version of each instrument must be saved, rather 
than just the individual changes. This must be a common pattern, not 
confined to investment banking. Any ideas how this pattern can be 
accommodated within CouchDB?

Calum Miller

Chris Anderson wrote:
> Calum,
> CouchDB should be easily able to handle this load.
> Please note that the built-in revision system is not designed for
> document history. Its sole purpose is to manage conflicting documents
> that result from edits done in separate copies of the DB, which are
> subsequently replicated into a single DB.
> If you allow CouchDB to create a new document for each daily import of
> each security, and create a view which makes these documents available
> by security and date, you should be able to access securities history
> fairly simply.
> Chris
> On Sat, Sep 13, 2008 at 12:31 PM, Calum Miller <> wrote:
>> Hi,
>> I trying to evaluate CouchDB for use within investment banking, yes some of
>> these banks still exist. I want to load 500,000 bonds into the database with
>> each bond containing around 100 fields. I would be looking to bulk load a
>> similar amount of these bonds every day whilst maintaining a history via the
>> revision feature. Are there any bulk load features available for CouchDB and
>> any tips on how to manage regular loads of this volume?
>> Many thanks in advance and best of luck with this project.
>> Calum Miller

View raw message