Return-Path: 
 <user-return-21173-apmail-couchdb-user-archive=couchdb.apache.org@couchdb.apache.org>
X-Original-To: apmail-couchdb-user-archive@www.apache.org
Delivered-To: apmail-couchdb-user-archive@www.apache.org
Received: from mail.apache.org (hermes.apache.org [140.211.11.3])
	by minotaur.apache.org (Postfix) with SMTP id A3BA7CE54
	for <apmail-couchdb-user-archive@www.apache.org>;
 Mon, 11 Jun 2012 12:03:22 +0000 (UTC)
Received: (qmail 74262 invoked by uid 500); 11 Jun 2012 12:03:21 -0000
Delivered-To: apmail-couchdb-user-archive@couchdb.apache.org
Received: (qmail 73867 invoked by uid 500); 11 Jun 2012 12:03:20 -0000
Mailing-List: contact user-help@couchdb.apache.org; run by ezmlm
Precedence: bulk
List-Help: <mailto:user-help@couchdb.apache.org>
List-Unsubscribe: <mailto:user-unsubscribe@couchdb.apache.org>
List-Post: <mailto:user@couchdb.apache.org>
List-Id: <user.couchdb.apache.org>
Reply-To: user@couchdb.apache.org
Delivered-To: mailing list user@couchdb.apache.org
Received: (qmail 73837 invoked by uid 99); 11 Jun 2012 12:03:19 -0000
Received: from minotaur.apache.org (HELO minotaur.apache.org) (140.211.11.9)
    by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 11 Jun 2012 12:03:19 +0000
Received: from localhost (HELO [192.168.1.7]) (127.0.0.1)
  (smtp-auth username rnewson, mechanism plain)
  by minotaur.apache.org (qpsmtpd/0.29) with ESMTP;
 Mon, 11 Jun 2012 12:03:18 +0000
Content-Type: text/plain; charset=iso-8859-1
Mime-Version: 1.0 (Apple Message framework v1278)
Subject: Re: How to do Bulk-insert from Huge JSON File (460 MB)
From: Robert Newson <rnewson@apache.org>
In-Reply-To: 
 <CAKXTtKGegDyrrxwASqcnTbeZKZOMV=nNHRw3Ba+uXow70cPEQw@mail.gmail.com>
Date: Mon, 11 Jun 2012 13:03:16 +0100
Content-Transfer-Encoding: quoted-printable
Message-Id: <93C87D8E-D891-4C54-B7AC-A621A68A11C9@apache.org>
References: 
 <CAKXTtKGegDyrrxwASqcnTbeZKZOMV=nNHRw3Ba+uXow70cPEQw@mail.gmail.com>
To: user@couchdb.apache.org
X-Mailer: Apple Mail (2.1278)


-d will load the whole file into memory and also interpret it as ascii, =
which might make it invalid.

use -T <filename> instead.

B.

On 11 Jun 2012, at 12:29, Mohammad Prabowo wrote:

> Hi. I need to do bulk-insert of document in my CouchDB database.
> I'm trying to follow the manual here:
> http://wiki.apache.org/couchdb/HTTP_Bulk_Document_API
>=20
> Here is my code:
>=20
> ~$ DB=3D"http://localhost:5984/employees"
> ~$ curl -H "Content-Type:application/json" -d =
@employees_selfContained.json
> -vX POST $DB/_bulk_docs
>=20
> the file employees_selfContained.json is a huge file =3D 465 MB. I've
> validated it using JSONLint and found nothing wrong
> Here's the curl's verbose output:
>=20
> curl -H "Content-Type:application/json" -d =
@employees_selfContained.json
> -vX POST $DB/_bulk_docs
> * About to connect() to 127.0.0.1 port 5984 (#0)
> * Trying 127.0.0.1... connected
> * Connected to 127.0.0.1 (127.0.0.1) port 5984 (#0)
>> POST /employees/_bulk_docs HTTP/1.1
>> User-Agent: curl/7.19.7 (i486-pc-linux-gnu) libcurl/7.19.7 =
OpenSSL/0.9.8k
> zlib/1.2.3.3 libidn/1.15
>> Host: 127.0.0.1:5984
>> Accept: */*
>> Content-Type:application/json
>> Content-Length: 439203931
>> Expect: 100-continue
>>=20
> < HTTP/1.1 100 Continue
> * Empty reply from server
> * Connection #0 to host 127.0.0.1 left intact
> curl: (52) Empty reply from server
> * Closing connection #0
>=20
> How can i do bulk-insert from that Huge single file? I prefer not to =
split
> the file into smaller size if possible..