Mailing-List: contact user-help@couchdb.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@couchdb.apache.org
Received-SPF: pass (athena.apache.org: domain of tisdall@gmail.com designates
 209.85.215.52 as permitted sender)
MIME-Version: 1.0
In-Reply-To: 
 <CAGxYWpYnxV5r6tn03dkq9fxf84yp9fnYACZr5jkaD3sUc-huGA@mail.gmail.com>
References: 
 <CAGxYWpZh_pNz3gH-WW544M3h8+=nLFYQo8vipAQBOpaN2CJJGQ@mail.gmail.com>
	<CAJ_m3YAMRFjVrCROUNK05hTqXtCb7ieZ3ED5D9-G6RF+G+v7Eg@mail.gmail.com>
	<CAGxYWpYnxV5r6tn03dkq9fxf84yp9fnYACZr5jkaD3sUc-huGA@mail.gmail.com>
Date: Thu, 16 Aug 2012 11:16:19 -0400
Message-ID: 
 <CAGxYWpbVp9C+kmRviGE00gswa+S5NgSUE73nUiroD7+ibcB2ow@mail.gmail.com>
Subject: Re: couchdb returning empty response
From: Tim Tisdall <tisdall@gmail.com>
To: user@couchdb.apache.org
Content-Type: text/plain; charset=ISO-8859-1

I tried running my code again and got a bunch of these entries in my logs:

[Thu, 16 Aug 2012 14:53:03 GMT] [error] [<0.20.0>] {error_report,<0.9.0>,
                                 {<0.20.0>,std_error,
                                  "File operation error: eacces.
Target: ./mochiweb_response.beam. Function: get_file. Process:
code_server."}}

here's the file listing for the file (as you can see it's readable by all):

-rw-r--r-- 1 root staff 1420 Aug  7 21:11
/usr/local/lib/couchdb/erlang/lib/mochiweb-1.4.1/ebin/mochiweb_response.beam


On Thu, Aug 16, 2012 at 10:27 AM, Tim Tisdall <tisdall@gmail.com> wrote:
> Okay, I'm completely unfamiliar with gdb but I tried and failed to get
> the stacktraces.  Here's what happened...
>
> I'm able to do everything up to attaching to the process and then
> things go wrong...
>
> (gdb) attach 29967
> Attaching to process 29967
> Reading symbols from /usr/lib/erlang/erts-5.8/bin/beam.smp...(no
> debugging symbols found)...done.
> Reading symbols from /lib/libutil.so.1...(no debugging symbols found)...done.
> Loaded symbols for /lib/libutil.so.1
> Reading symbols from /lib/libdl.so.2...(no debugging symbols found)...done.
> Loaded symbols for /lib/libdl.so.2
> Reading symbols from /lib/libm.so.6...(no debugging symbols found)...done.
> Loaded symbols for /lib/libm.so.6
> Reading symbols from /lib/libncurses.so.5...(no debugging symbols found)...done.
> Loaded symbols for /lib/libncurses.so.5
> Reading symbols from /lib/libpthread.so.0...(no debugging symbols found)...done.
>    [ *SNIP* ]
> Reading symbols from /usr/lib/libz.so.1...(no debugging symbols found)...done.
> Loaded symbols for /usr/lib/libz.so.1
> Reading symbols from
> /usr/local/lib/couchdb/erlang/lib/snappy-1.0.3/priv/snappy_nif.so...done.
> Loaded symbols for
> /usr/local/lib/couchdb/erlang/lib/snappy-1.0.3/priv/snappy_nif.so
> 0x00007f01e153c3e3 in select () from /lib/libc.so.6
>
>
> If I try to see the process with ps in another terminal it now says:
> 29967 pts/3    Zl   0:00 [beam.smp] <defunct>
>
> At this point couchdb is no longer responding so I'm not able to run
> my script to try to get that stack trace.  I also tried typing
> "continue" into gdb and the process stays "defunct".
>
> Do I need to install the debugging versions of all those libraries in
> order for this to work?
>
> -Tim
>
> On Tue, Aug 14, 2012 at 11:41 PM, Paul Davis
> <paul.joseph.davis@gmail.com> wrote:
>> On Tue, Aug 14, 2012 at 9:38 PM, Tim Tisdall <tisdall@gmail.com> wrote:
>>> I'm still having problems with couchdb, but I'm trying out different
>>> things to see if I can narrow down what the problem is...
>>>
>>> I stopped using fsockopen() in PHP and am using curl now to hopefully
>>> be able to see more debugging info.
>>>
>>> I get an empty response when sending a POST to _bulk_docs.  From the
>>> couch logs it seems like the server restarts in the middle of
>>> processing the request.  Here's what I have in my logs:  (I have no
>>> idea what the _replicator portion is about there, I'm currently not
>>> using it)
>>>
>>>
>>> [Wed, 15 Aug 2012 02:27:30 GMT] [debug] [<0.1255.0>] 'POST'
>>> /app_stats_test/_bulk_docs {1,0} from "127.0.0.1"
>>> Headers: [{'Accept',"*/*"},
>>>           {'Content-Length',"2802300"},
>>>           {'Content-Type',"application/json"},
>>>           {'Host',"localhost:5984"}]
>>> [Wed, 15 Aug 2012 02:27:30 GMT] [debug] [<0.1255.0>] OAuth Params: []
>>> [Wed, 15 Aug 2012 02:27:45 GMT] [debug] [<0.115.0>] Include Doc:
>>> <<"_design/_replicator">> {1,
>>>                                                             <<91,250,44,153,
>>>                                                               238,254,43,46,
>>>                                                               180,150,45,181,
>>>                                                               10,163,207,212>>}
>>> [Wed, 15 Aug 2012 02:27:45 GMT] [info] [<0.32.0>] Apache CouchDB has
>>> started on http://127.0.0.1:5984/
>>>
>>>
>>> In my code logs I have the following by running curl in verbose mode:
>>>
>>> * About to connect() to localhost port 5984 (#0)
>>> *   Trying 127.0.0.1... * connected
>>> * Connected to localhost (127.0.0.1) port 5984 (#0)
>>>> POST /app_stats_test/_bulk_docs HTTP/1.0
>>> Host: localhost:5984
>>> Accept: */*
>>> Content-Type: application/json
>>> Content-Length: 2802300
>>>
>>> * Empty reply from server
>>> * Connection #0 to host localhost left intact
>>> curl error: 52 : Empty reply from server
>>>
>>>
>>>
>>> I also tried using HTTP/1.1 and I get an empty response after
>>> receiving only a "100 Continue", but the end result appears the same.
>>>
>>> -Tim
>>
>> If you have a request that triggers this, a good way to catch it is like such:
>>
>>     $ /usr/local/bin/couchdb # or however you start it
>>     $ ps ax | grep beam.smp # Get the pid of couchdb
>>     $ gdb
>>        (gdb) attach $pid # Where $pid was just found with ps. Might
>> throw up an access prompt
>>        (gdb) continue
>>        # At this point, run the command that makes couchdb reboot in a
>>        # different console. If it happens you should see Gdb notice the
>>        # error. Then the following:
>>        (gdb) t a a bt
>>
>> And that should spew out a bunch of stack traces. If you can get that
>> we should be able to fairly specifically narrow down the issue.