couchdb-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jeff Hinrichs (JIRA)" <j...@apache.org>
Subject [jira] Commented: (COUCHDB-270) Replication w/ Large Attachments Fails
Date Sat, 28 Feb 2009 17:04:12 GMT

    [ https://issues.apache.org/jira/browse/COUCHDB-270?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12677694#action_12677694
] 

Jeff Hinrichs commented on COUCHDB-270:
---------------------------------------

The above failure was created using futon.  

I am trying to build a script to replicate the failure on demand but I am seeing something
different now, the error is kind of like couchdb-240 (see below for more error output)  I
will attach my script to this issue.  Hopefully it will help.

--error information---

[Sat, 28 Feb 2009 16:56:35 GMT] [info] [<0.962.0>] 192.168.2.52 - - 'POST' /couchdb-270-target/_bulk_docs
201

[Sat, 28 Feb 2009 16:56:49 GMT] [debug] [<0.979.0>] 'POST' /couchdb-270-target/_bulk_docs
{1,1}
Headers: [{'Host',"192.168.2.194:5984"},
          {'Transfer-Encoding',"chunked"}]

[Sat, 28 Feb 2009 16:57:18 GMT] [error] [<0.906.0>] ** Generic server <0.906.0>
terminating 
** Last message in was {update_docs,[[],[],[],[],[],[],[],[],[],[],[],[],[],
                                     [],[],[],[],[],[],[],[],[],[],[],[],[],
                                     [],[],[],[],[],[],[],[],[],[],[],[],[],
                                     [],[],[],[],[],[],[],[],[],[],[],[],[],
                                     [],[],[],[],[],[],[],[],[],[],[],[],[],
                                     [],[],[],[],[],[],[],[],[],[],[],[],[],
                                     [],[],[],[],[],[],[],[],[],[],[],[],[],
                                     [],[],[],[],[],[],[],[],[]],
                                    []}
** When Server state == {db,<0.905.0>,<0.906.0>,nil,<<"1235840109444771">>,
                            <0.904.0>,<0.908.0>,
                            {db_header,0,100,
                                {104867784,0},
                                {104876441,{100,0}},
                                {104872023,100},
                                nil,0,nil,nil},
                            {stream,<0.907.0>,<0.904.0>},
                            {btree,<0.904.0>,
                                {104876441,{100,0}},
                                #Fun<couch_db_updater.6.5983005>,
                                #Fun<couch_db_updater.7.77407758>,
                                #Fun<couch_db_updater.5.41936925>,
                                #Fun<couch_db_updater.8.15562944>},
                            {btree,<0.904.0>,
                                {104872023,100},
                                #Fun<couch_db_updater.9.64364170>,
                                #Fun<couch_db_updater.10.113053104>,
                                #Fun<couch_btree.5.112258129>,
                                #Fun<couch_db_updater.11.146363>},
                            {btree,<0.904.0>,nil,
                                #Fun<couch_btree.0.23070627>,
                                #Fun<couch_btree.1.117278773>,
                                #Fun<couch_btree.2.112258129>,nil},
                            100,<<"couchdb-270-target">>,
                            "/usr/local/var/lib/couchdb/couchdb-270-target.couch",
                            [],[],nil,
                            {user_ctx,null,[]},
                            nil}
** Reason for termination == 
** {function_clause,
       [{couch_db_updater,'-update_docs_int/3-fun-0-',[[],{[],[]}]},
        {lists,foldl,3},
        {couch_db_updater,update_docs_int,3},
        {couch_db_updater,handle_call,3},
        {gen_server,handle_msg,5},
        {proc_lib,init_p,5}]}


[Sat, 28 Feb 2009 16:57:19 GMT] [error] [<0.906.0>] {error_report,<0.22.0>,
    {<0.906.0>,crash_report,
     [[{pid,<0.906.0>},
       {registered_name,[]},
       {error_info,
           {exit,
               {function_clause,
                   [{couch_db_updater,'-update_docs_int/3-fun-0-',
                        [[],{[],[]}]},
                    {lists,foldl,3},
                    {couch_db_updater,update_docs_int,3},
                    {couch_db_updater,handle_call,3},
                    {gen_server,handle_msg,5},
                    {proc_lib,init_p,5}]},
               [{gen_server,terminate,6},{proc_lib,init_p,5}]}},
       {initial_call,
           {gen,init_it,
               [gen_server,<0.905.0>,<0.905.0>,couch_db_updater,
                {<0.905.0>,<<"couchdb-270-target">>,
                 "/usr/local/var/lib/couchdb/couchdb-270-target.couch",
                 <0.904.0>,
                 [create,{user_ctx,{user_ctx,null,[<<"_admin">>]}}]},
                []]}},
       {ancestors,
           [<0.905.0>,couch_server,couch_primary_services,
            couch_server_sup,<0.1.0>]},
       {messages,[]},
       {links,[<0.905.0>,<0.907.0>]},
       {dictionary,[]},
       {trap_exit,false},
       {status,running},
       {heap_size,6765},
       {stack_size,23},
       {reductions,96260}],
      [{neighbour,
           [{pid,<0.907.0>},
            {registered_name,[]},
            {initial_call,
                {gen,init_it,
                    [gen_server,<0.906.0>,<0.906.0>,couch_stream,
                     {{0,0},<0.904.0>},
                     []]}},
            {current_function,{gen_server,loop,6}},
            {ancestors,
                [<0.906.0>,<0.905.0>,couch_server,couch_primary_services,
                 couch_server_sup,<0.1.0>]},
            {messages,[]},
            {links,[<0.906.0>]},
            {dictionary,[]},
            {trap_exit,false},
            {status,waiting},
            {heap_size,233},
            {stack_size,12},
            {reductions,5849}]},
       {neighbour,
           [{pid,<0.905.0>},
            {registered_name,[]},
            {initial_call,
                {gen,init_it,
                    [gen_server,<0.44.0>,<0.44.0>,couch_db,
                     {<<"couchdb-270-target">>,
                      "/usr/local/var/lib/couchdb/couchdb-270-target.couch",
                      <0.904.0>,
                      [create,{user_ctx,{user_ctx,null,[<<"_admin">>]}}]},
                     []]}},
            {current_function,{gen_server,loop,6}},
            {ancestors,
                [couch_server,couch_primary_services,couch_server_sup,
                 <0.1.0>]},
            {messages,[]},
            {links,[<0.44.0>,<0.906.0>]},
            {dictionary,[]},
            {trap_exit,false},
            {status,waiting},
            {heap_size,987},
            {stack_size,12},
            {reductions,320}]}]]}}

[Sat, 28 Feb 2009 16:57:19 GMT] [error] [<0.979.0>] Uncaught error in HTTP request:
{exit,
                                 {{function_clause,
                                   [{couch_db_updater,
                                     '-update_docs_int/3-fun-0-',
                                     [[],{[],[]}]},
                                    {lists,foldl,3},
                                    {couch_db_updater,update_docs_int,3},
                                    {couch_db_updater,handle_call,3},
                                    {gen_server,handle_msg,5},
                                    {proc_lib,init_p,5}]},
                                  {gen_server,call,
                                   [<0.906.0>,
                                    {update_docs,
                                     [[],[],[],[],[],[],[],[],[],[],[],[],[],
                                      [],[],[],[],[],[],[],[],[],[],[],[],[],
                                      [],[],[],[],[],[],[],[],[],[],[],[],[],
                                      [],[],[],[],[],[],[],[],[],[],[],[],[],
                                      [],[],[],[],[],[],[],[],[],[],[],[],[],
                                      [],[],[],[],[],[],[],[],[],[],[],[],[],
                                      [],[],[],[],[],[],[],[],[],[],[],[],[],
                                      [],[],[],[],[],[],[],[],[]],
                                     []},
                                    infinity]}}}

[Sat, 28 Feb 2009 16:57:19 GMT] [debug] [<0.979.0>] Stacktrace: [{gen_server,call,3},
             {couch_db,write_and_commit,3},
             {couch_httpd_db,db_req,2},
             {couch_httpd_db,do_db_req,2},
             {couch_httpd,handle_request,3},
             {mochiweb_http,headers,5},
             {proc_lib,init_p,5}]

[Sat, 28 Feb 2009 16:57:19 GMT] [debug] [<0.979.0>] HTTPd 500 error response:
 {"error":"{function_clause,[{couch_db_updater,'-update_docs_int/3-fun-0-',\n            
                       [[],{[],[]}]},\n                  {lists,foldl,3},\n              
   {couch_db_updater,update_docs_int,3},\n                  {couch_db_updater,handle_call,3},\n
                 {gen_server,handle_msg,5},\n                  {proc_lib,init_p,5}]}","reason":"{gen_server,call,\n
           [<0.906.0>,\n             {update_docs,[[],[],[],[],[],[],[],[],[],[],[],[],[],[],[],\n
                          [],[],[],[],[],[],[],[],[],[],[],[],[],[],[],[],[],\n          
                [],[],[],[],[],[],[],[],[],[],[],[],[],[],[],[],[],\n                    
      [],[],[],[],[],[],[],[],[],[],[],[],[],[],[],[],[],\n                           [],[],[],[],[],[],[],[],[],[],[],[],[],[],[],[],[],\n
                          [],[],[],[],[],[],[],[],[],[],[],[],[],[],[],[],[]],\n         
                []},\n             infinity]}"}



> Replication w/ Large Attachments Fails
> --------------------------------------
>
>                 Key: COUCHDB-270
>                 URL: https://issues.apache.org/jira/browse/COUCHDB-270
>             Project: CouchDB
>          Issue Type: Bug
>          Components: Database Core
>    Affects Versions: 0.9
>         Environment: Apache CouchDB 0.9.0a748379
>            Reporter: Jeff Hinrichs
>
> Attempting to replicate a database with largish attachments (<= ~18MB of attachments
in a doc, less thatn 200 docs)  from one machine to another fails consistently and at the
same point.
> Scenario:
> Both servers are running from HEAD and I've been tracking for some time.  This problem
has been around as long as I've been using couch.
> Machine A holds the original database, Machine B is the server that is doing a PULL replication
> During the replication, Machine A starts showing the following sporadically in the log:
> [Fri, 27 Feb 2009 14:02:48 GMT] [debug] [<0.5902.3>] 'GET'
> /delasco-invoices/INV00652429?revs=true&attachments=true&latest=true&open_revs=["425644723"]
> {1,
>                             1}
> Headers: [{'Host',"192.168.2.52:5984"}]
> [Fri, 27 Feb 2009 14:02:48 GMT] [error] [<0.5901.3>] Uncaught error in
> HTTP request: {exit,normal}
> [Fri, 27 Feb 2009 14:02:48 GMT] [debug] [<0.5901.3>] Stacktrace:
> [{mochiweb_request,send,2},
>             {couch_httpd,send_chunk,2},
>             {couch_httpd_db,db_doc_req,3},
>             {couch_httpd_db,do_db_req,2},
>             {couch_httpd,handle_request,3},
>             {mochiweb_http,headers,5},
>             {proc_lib,init_p,5}]
> [Fri, 27 Feb 2009 14:02:48 GMT] [debug] [<0.5901.3>] HTTPd 500 error response:
>  {"error":"error","reason":"normal"}
> As the replication continues, the frequency of these error "Uncaught error in HTTP request:
{exit,normal}"  increase.  Until the error is being constantly repeated.  Then Machine B stops
sending requests, no more log output, no errors, the last thing in Machine B's log file is:
> [Fri, 27 Feb 2009 14:03:24 GMT] [info] [<0.20893.1>] retrying
> couch_rep HTTP get request due to {error, req_timedout}: [104,116,
>                                                                   116,112,58,
>                                                                   47,47,49,
>                                                                   57,50,46,
>                                                                   49,54,56,
>                                                                   46,50,46,
>                                                                   53,50,58,
>                                                                   53,57,56,
>                                                                   52,47,100,
>                                                                   101,108,97,
>                                                                   115,99,111,
>                                                                   45,105,110,
>                                                                   118,111,
>                                                                   105,99,101,
>                                                                   115,47,73,
>                                                                   78,86,48,
>                                                                   48,54,53,
>                                                                   50,49,51,
>                                                                   56,63,114,
>                                                                   101,118,
>                                                                   115,61,116,
>                                                                   114,117,
>                                                                   101,38,97,
>                                                                   116,116,97,
>                                                                   99,104,109,
>                                                                   101,110,
>                                                                   116,115,61,
>                                                                   116,114,
>                                                                   117,101,38,
>                                                                   108,97,116,
>                                                                   101,115,
>                                                                   116,61,116,
>                                                                   114,117,
>                                                                   101,38,111,
>                                                                   112,101,
>                                                                   110,95,114,
>                                                                   101,118,
>                                                                   115,61,91,
>                                                                   34,
> <<"3070455362">>,
>                                                                   34,93]
> A request for status from the couchdb init.d script returns nothing and checking the
processes returns:
> (demo-couchdb)jlh@mars:~/projects/venvs/demo-couchdb/src$ ps ax|grep cou
> 29281 pts/2    S+     0:00 grep cou
> (demo-couchdb)jlh@mars:~/projects/venvs/demo-couchdb/src$ ps ax|grep beam
> 29305 pts/2    R+     0:00 grep beam
> In fact, couch has gone away completely on Machine B.  In fact, couch's death is so quick
it can't even say why.
> Attempts to incrementally replicate after the first failure die at exactly the same place.
> I can replicate this same database on the same machine from one database to another without
issue.  I can dump and reload the database with no problems.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message