couchdb-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Monica Razdan <mon...@almende.org>
Subject cUrl _bulk_docs upload runs out of memory
Date Tue, 30 Aug 2011 08:35:19 GMT
Hello,

I am exporting data from mysql to couchdb using _bulk_docs. Some of the json
files are in gigs. When I try to bulk upload docs curl runs out of memory
reading these files. Is there a way to stream the files while sending a post
request? For now I'm splitting the bigger files  but then some of them still
end up as big as 800mb because some of the file have like 2,40,000 of
rowset. What is the max size limit when posting files using curl ? Is there
a alternate to curl for uploading chunk of json files to couchdb ?

Thanks,
Monica

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message