couchdb-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Simon Woodhead <simon.woodh...@simwood.com>
Subject Map/reduce and transactions
Date Fri, 13 Aug 2010 14:55:59 GMT
Hi folks

As we explore more ways to migrate to CouchDB we're exploring alternatives to transactions.
The books shows a bank where transfers are stored as a single document with the balance being
the result of a map/reduce function. That makes sense. 

For us, our scale is tipped differently in that we have hundreds of millions of tiny transactions
affecting relatively few balances. In MySQL (ironically) we denormalise this and hold balances
in their own table but then can insert transactions and update balances within a single transaction.


Looking at moving this to CouchDB would mean getting rid of the balances table and just using
a map/reduce function. I recognise that a given document will only be handled once and that
this is therefore more efficient than it may seem to a SQL jock like me but I wanted to ask
about whether it truly scales at volume. 

I guess I'm asking whether whatever index the view creates contains a reference to every document
(and thus gets bigger with more documents) or just contains the output and the _id of the
last document processed. I can see the first one running into issues quickly whilst the second
would seem to scale indefinitely. 

FWIW it takes over a day to compute the sum against the table in MySQL as the table is being
constantly appended - hence why we denormalised it! It therefore feels really strange to normalise
more when moving to a document store so any advice is welcome! 

Thanks
Simon--
***** Email confidentiality notice *****

This message is private and confidential. If you have received this message in error, please
notify us and remove it from your system.


Simwood eSMS Limited is a limited company registered in England and Wales. Registered number:
03379831. Registered office: c/o HW Chartered Accountants, Keepers Lane, The Wergs, Wolverhampton,
WV6 8UA. Trading address: Falcon Drive, Cardiff Bay, Cardiff, CF10 4RU.



Mime
View raw message