couchdb-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ryan Richt (JIRA)" <>
Subject [jira] [Commented] (COUCHDB-1490) Problems with views on large documents JSONs
Date Fri, 08 Jun 2012 13:57:23 GMT


Ryan Richt commented on COUCHDB-1490:


I'm guessing you dont need to do any M/R indexing over the protein structure description (the
model / atoms)  that make up most of the document.

If this is true, move all of that su-tree of the JSON to a binary attachment. You can't M/R
over it, and the doc will be about the same size, but the amount of data the view indexer
has to pack/unpack will be greatly reduced and your problem should go away. I know that's
not a real solution, but it is more in-line with the intended use cases of CouchDB.

we've seen documents work best when the JSON is a few kB to a few MB, but attachments can
be GB range without issues.
> Problems with views on large documents JSONs
> --------------------------------------------
>                 Key: COUCHDB-1490
>                 URL:
>             Project: CouchDB
>          Issue Type: Bug
>    Affects Versions: 1.2
>         Environment: Mac Os x 10.6.8, intel Architecture (x86_64), 8Gb of Ram, Erlang
R15B01 (erts-5.9.1)
>            Reporter: Francesco
> Hi,
> i run a couchdb server (v1.2.0) over a mac (intel architecture, 8gb of ram,
> os x version 10.6.8) installed with brew.
> The server itself is used as a storage of big jsons (example:
and ) for
a tiny uni project.
> When we load more than 3 of these jsons, all the map functions (we created to retrieve
documents besides a simple get by id) does not work.
> A typical map is:
> function(doc){if(doc.TITLE.title.match('.*INSULIN.*') !== null) emit(doc.ID,
> doc);}
> but even a
> function(doc){emit(doc.ID, doc.ID)}
> cease to work.
> while when there are just 3 or 2 jsons in the database they work just fine. I tried increasing
the stack for couchjs (1gb now, going over 1gb doesn't work it seems), increasing limits for
files (4096), increasing timeout for processes but in the end i don't get any results and
only a (Error:
> os_process_error {exit_status,0}) from the db.

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:!default.jspa
For more information on JIRA, see:


View raw message