incubator-couchdb-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jan Lehnardt (JIRA)" <j...@apache.org>
Subject [jira] Updated: (COUCHDB-784) Add non-buffering bulk_docs request API
Date Wed, 02 Jun 2010 18:17:43 GMT

     [ https://issues.apache.org/jira/browse/COUCHDB-784?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Jan Lehnardt updated COUCHDB-784:
---------------------------------

        Summary: Add non-buffering bulk_docs request API  (was: erlang crash on bulk upload
of lots of documents)
     Issue Type: Improvement  (was: Bug)
       Priority: Minor  (was: Major)
    Component/s: Database Core
                 HTTP Interface

This is standard Erlang behaviour and works as intended.

Erlang's philosophy is to crash hard and fast in error situations. If no more ram is available,
what's a VM to do? Erlang & CouchDB are set up to restart instantly to avoid unnecessary
downtimes (see `couchdb -i` for more information).

--

The particular solution would be to reduce the bulk request until it doesn't exceed ram.

--

Alternatively, we'd have to introduce a non-buffering JSON parser.

Renaming the issue to track that (future) progress.

> Add non-buffering bulk_docs request API
> ---------------------------------------
>
>                 Key: COUCHDB-784
>                 URL: https://issues.apache.org/jira/browse/COUCHDB-784
>             Project: CouchDB
>          Issue Type: Improvement
>          Components: Database Core, HTTP Interface
>    Affects Versions: 0.11
>         Environment: windows 32 vista
>            Reporter: joe pribele
>            Priority: Minor
>         Attachments: erl_crash.dump, json.zip
>
>
> Uploading 10,000 largish documents to the _bulk_docs results in a erlang crash trying
to allocate a lot of heap memory

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message