couchdb-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Adam Kocoloski <kocol...@apache.org>
Subject Re: Documents with large numbers of fields
Date Thu, 08 Oct 2009 02:26:34 GMT
On Oct 7, 2009, at 5:05 PM, Mike Leddy wrote:

> Hi all,
>
> I have a database with approx 100 million records that I like to port
> to a couchdb cluster. The documents that will be inserted will
> vary between 150-250 fields each.
>
> (I will be doing some cool/complex stuff with views for recently
> inserted data in the 24-48 hour range)
>
> A series of historical reports are generated from this data that
> are selected  using a simple criterion ie. an interval in time
> together with multiple locations BUT the fields that are used vary
> significantly depending on the reports (and combination of reports)
> that are generated simultaneously.
>
> If I were to use views for each report wouldn't I be building a
> index plus the resulting field values for each possible report
> on disk ?
>
> If I use only one view I will save disk but returning the full
> documents via HTTP which will be impractically slow :-(
>
> Is there any way I can have one index being used but filter the
> fields being returned in the documents ? I guess conceptually its
> single 'view' combined with different 'shows' that return JSON.
>
> Have I missed some obvious way to do this ?
>
> TIA
>
> Mike

Hi Mike, I think you're right, the way to accomplish this is to write  
multiple 'list' functions that send different subsets of the value  
stored in your view.  The API for _list is quite different in 0.9 and  
0.10, see

http://wiki.apache.org/couchdb/Formatting_with_Show_and_List

Best, Adam


Mime
View raw message