couchdb-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Martin Hewitt <mar...@thenoi.se>
Subject Re: Performance of many documents vs large documents
Date Wed, 11 Jan 2012 02:34:53 GMT
Very useful advice everyone, thanks.

Martin 

On 11 Jan 2012, at 01:42, Simon Metson wrote:

> Another thing that might be useful is having 'rotating' databases. E.g. for today things
are stored at the second level, for the proceeding week things are stored at the 30s level,
for last week things are stored at the 5 min level etc. This can be accomplished by hitting
an appropriate view and dumping the result into a new database. It works if you're interested
in aggregated data over the time period, if you always need the second-level granularity then
it's (obviously) not so hot.  
> Cheers
> Simon
> 
> 
> On Tuesday, 10 January 2012 at 23:50, Rogutės Sparnuotos wrote:
> 
>> Mark Hahn (2012-01-10 15:08):
>>>> I could have one document per metric, leading to a small number of
>>>> documents, but with each document containing ticks for every 5-second
>>>> interval of any given day, these documents would quickly become huge.
>>>> 
>>> 
>>> 
>>> This would not only make the docs large, but you'd have a trail of huge
>>> outdated docs. You would get n-squared storage problems quickly. Remember
>>> that all versions of a doc are stored until you do a compaction.
>>> 
>> 
>> 
>> Usually metrics don't need to be updated. If this is the case, then there
>> should be no storage problems with huge docs (compared to tiny ones).
>> 
>>> I'd suggest one doc per second containing all the metric values for that
>>> second.
>>> 
>> 
>> 
>> I would say do not dump more data than you need, i.e. don't use 1s
>> intervals if 5s is enough for you application.
>> 
>> Regarding doc size, my guess is that CouchDB's architecture does not care,
>> and the decision must be made based on intended usage, but the original
>> poster did not describe how he is going to use the data (what kind of
>> views, list and show functions will be needed).
>> 
>> --  
>> -- Rogutės Sparnuotos
>> 
>> 
> 
> 


Mime
View raw message