couchdb-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Frank Röhm <francwal...@gmx.net>
Subject Compaction doesn't work on big db - unknown_error, reason: function_clause
Date Sat, 14 Mar 2015 07:51:34 GMT
Hello

On my Ubuntu 12.04 LTS server I have installed couchdb version 1.6.1-0ubuntu1.
$ couchdb -V 
Apache CouchDB 1.6.1

I have “scraped” the german wiki into this couchdb and now I have a uncompressed 44 GB
database “de-m-wikipedia-org”, which I try to compact with:

$ curl -H "Content-Type: application/json" -X POST http://localhost:5984/de-m-wikipedia-org/_compact
{"ok":true}

But this throws an error: 

Error in process <0.247.0> with exit value: {function_clause,[{couch_compress,decompress,[<<0
bytes>>]},{couch_file,pread_term,2},{couch_btree,get_node,2},{couch_btree,stream_node,7},{couch_btree,stream_kp_node,7},{couch_btree,stream_kp_node,8},{couch_btree,fold,4},{couch_db_updater...


complete error is attached in the end of this mail.

and I get a file de-m-wikipedia-org.couch.compact which has only 79 Byte and wouldn’t grow
bigger.

If I compact another db, which had 7.4 GB, the compaction works flawless, I read in the log:

[Sat, 14 Mar 2015 07:37:21 GMT] [info] [<0.695.0>] Starting compaction for db "de-m-wiktionary-org"
[Sat, 14 Mar 2015 07:37:21 GMT] [info] [<0.139.0>] 127.0.0.1 - - POST /de-m-wiktionary-org/_compact
202
[Sat, 14 Mar 2015 07:39:39 GMT] [info] [<0.695.0>] Compaction for db "de-m-wiktionary-org"
completed.

 and after compaction the file is 1.7 GB.

So I guess it is related to this big database.
Should I just delete it and restart to scrape again from scratch?


Thank you

frank






PS.: this is the complete error message in the couch.log


[Sat, 14 Mar 2015 07:28:35 GMT] [info] [<0.31.0>] Apache CouchDB has started on http://127.0.0.1:5984/
[Sat, 14 Mar 2015 07:30:05 GMT] [info] [<0.242.0>] Starting compaction for db "de-m-wikipedia-org"
[Sat, 14 Mar 2015 07:30:05 GMT] [info] [<0.137.0>] 127.0.0.1 - - POST /de-m-wikipedia-org/_compact
202
[Sat, 14 Mar 2015 07:30:05 GMT] [error] [emulator] Error in process <0.247.0> with exit
value: {function_clause,[{couch_compress,decompress,[<<0 bytes>>]},{couch_file,pread_term,2},{couch_btree,get_node,2},{couch_btree,stream_node,7},{couch_btree,stream_kp_node,7},{couch_btree,stream_kp_node,8},{couch_btree,fold,4},{couch_db_updater...



[Sat, 14 Mar 2015 07:30:05 GMT] [error] [<0.242.0>] ** Generic server <0.242.0>
terminating 
** Last message in was {'EXIT',<0.247.0>,
                           {function_clause,
                               [{couch_compress,decompress,[<<>>]},
                                {couch_file,pread_term,2},
                                {couch_btree,get_node,2},
                                {couch_btree,stream_node,7},
                                {couch_btree,stream_kp_node,7},
                                {couch_btree,stream_kp_node,8},
                                {couch_btree,fold,4},
                                {couch_db_updater,copy_compact,3}]}}
** When Server state == {db,<0.241.0>,<0.242.0>,<0.247.0>,
                            <<"1426318205344581">>,<0.243.0>,<0.239.0>,
                            <0.245.0>,
                            {db_header,6,2624284,0,
                                {46859286084,
                                 {1775313,0,15923938389},
                                 159502872},
                                {46859291358,1775313,123250096},
                                nil,0,nil,nil,1000},
                            2624284,
                            {btree,<0.239.0>,
                                {46859286084,
                                 {1775313,0,15923938389},
                                 159502872},
                                #Fun<couch_db_updater.10.55895019>,
                                #Fun<couch_db_updater.11.100913286>,
                                #Fun<couch_btree.5.296538>,
                                #Fun<couch_db_updater.12.39068440>,snappy},
                            {btree,<0.239.0>,
                                {46859291358,1775313,123250096},
                                #Fun<couch_db_updater.13.114276184>,
                                #Fun<couch_db_updater.14.2340873>,
                                #Fun<couch_btree.5.296538>,
                                #Fun<couch_db_updater.15.23651859>,snappy},
                            {btree,<0.239.0>,nil,
                                #Fun<couch_btree.3.127739064>,
                                #Fun<couch_btree.4.112272954>,
                                #Fun<couch_btree.5.296538>,nil,snappy},
                            2624284,<<"de-m-wikipedia-org">>,
                            "/var/lib/couchdb/de-m-wikipedia-org.couch",[],[],
                            nil,
                            {user_ctx,null,[],undefined},
                            nil,1000,
                            [before_header,after_header,on_file_open],
                            [{user_ctx,
                                 {user_ctx,null,
                                     [<<"_admin">>],
                                     <<"{couch_httpd_auth, default_authentication_handler}">>}}],
                            snappy,nil,nil}
** Reason for termination == 
** {function_clause,[{couch_compress,decompress,[<<>>]},
                     {couch_file,pread_term,2},
                     {couch_btree,get_node,2},
                     {couch_btree,stream_node,7},
                     {couch_btree,stream_kp_node,7},
                     {couch_btree,stream_kp_node,8},
                     {couch_btree,fold,4},
                     {couch_db_updater,copy_compact,3}]}

[Sat, 14 Mar 2015 07:30:05 GMT] [error] [<0.242.0>] {error_report,<0.30.0>,
                        {<0.242.0>,crash_report,
                         [[{initial_call,
                               {couch_db_updater,init,['Argument__1']}},
                           {pid,<0.242.0>},
                           {registered_name,[]},
                           {error_info,
                               {exit,
                                   {function_clause,
                                       [{couch_compress,decompress,[<<>>]},
                                        {couch_file,pread_term,2},
                                        {couch_btree,get_node,2},
                                        {couch_btree,stream_node,7},
                                        {couch_btree,stream_kp_node,7},
                                        {couch_btree,stream_kp_node,8},
                                        {couch_btree,fold,4},
                                        {couch_db_updater,copy_compact,3}]},
                                   [{gen_server,terminate,6},
                                    {proc_lib,init_p_do_apply,3}]}},
                           {ancestors,[<0.241.0>,<0.238.0>]},
                           {messages,[]},
                           {links,[<0.241.0>]},
                           {dictionary,[]},
                           {trap_exit,true},
                           {status,running},
                           {heap_size,987},
                           {stack_size,24},
                           {reductions,1753}],
                          []]}}
[Sat, 14 Mar 2015 07:30:05 GMT] [error] [<0.241.0>] ** Generic server <0.241.0>
terminating 
** Last message in was {'EXIT',<0.242.0>,
                           {function_clause,
                               [{couch_compress,decompress,[<<>>]},
                                {couch_file,pread_term,2},
                                {couch_btree,get_node,2},
                                {couch_btree,stream_node,7},
                                {couch_btree,stream_kp_node,7},
                                {couch_btree,stream_kp_node,8},
                                {couch_btree,fold,4},
                                {couch_db_updater,copy_compact,3}]}}
** When Server state == {db,<0.241.0>,<0.242.0>,<0.247.0>,
                            <<"1426318205344581">>,<0.243.0>,<0.239.0>,
                            <0.245.0>,
                            {db_header,6,2624284,0,
                                {46859286084,
                                 {1775313,0,15923938389},
                                 159502872},
                                {46859291358,1775313,123250096},
                                nil,0,nil,nil,1000},
                            2624284,
                            {btree,<0.239.0>,
                                {46859286084,
                                 {1775313,0,15923938389},
                                 159502872},
                                #Fun<couch_db_updater.10.55895019>,
                                #Fun<couch_db_updater.11.100913286>,
                                #Fun<couch_btree.5.296538>,
                                #Fun<couch_db_updater.12.39068440>,snappy},
                            {btree,<0.239.0>,
                                {46859291358,1775313,123250096},
                                #Fun<couch_db_updater.13.114276184>,
                                #Fun<couch_db_updater.14.2340873>,
                                #Fun<couch_btree.5.296538>,
                                #Fun<couch_db_updater.15.23651859>,snappy},
                            {btree,<0.239.0>,nil,
                                #Fun<couch_btree.3.127739064>,
                                #Fun<couch_btree.4.112272954>,
                                #Fun<couch_btree.5.296538>,nil,snappy},
                            2624284,<<"de-m-wikipedia-org">>,
                            "/var/lib/couchdb/de-m-wikipedia-org.couch",[],[],
                            nil,
                            {user_ctx,null,[],undefined},
                            nil,1000,
                            [before_header,after_header,on_file_open],
                            [{user_ctx,
                                 {user_ctx,null,
                                     [<<"_admin">>],
                                     <<"{couch_httpd_auth, default_authentication_handler}">>}}],
                            snappy,nil,nil}
** Reason for termination == 
** {function_clause,[{couch_compress,decompress,[<<>>]},
                     {couch_file,pread_term,2},
                     {couch_btree,get_node,2},
                     {couch_btree,stream_node,7},
                     {couch_btree,stream_kp_node,7},
                     {couch_btree,stream_kp_node,8},
                     {couch_btree,fold,4},
                     {couch_db_updater,copy_compact,3}]}

[Sat, 14 Mar 2015 07:30:05 GMT] [error] [<0.241.0>] {error_report,<0.30.0>,
                        {<0.241.0>,crash_report,
                         [[{initial_call,{couch_db,init,['Argument__1']}},
                           {pid,<0.241.0>},
                           {registered_name,[]},
                           {error_info,
                               {exit,
                                   {function_clause,
                                       [{couch_compress,decompress,[<<>>]},
                                        {couch_file,pread_term,2},
                                        {couch_btree,get_node,2},
                                        {couch_btree,stream_node,7},
                                        {couch_btree,stream_kp_node,7},
                                        {couch_btree,stream_kp_node,8},
                                        {couch_btree,fold,4},
                                        {couch_db_updater,copy_compact,3}]},
                                   [{gen_server,terminate,6},
                                    {proc_lib,init_p_do_apply,3}]}},
                           {ancestors,[<0.238.0>]},
                           {messages,[]},
                           {links,[<0.88.0>]},
                           {dictionary,[]},
                           {trap_exit,true},
                           {status,running},
                           {heap_size,987},
                           {stack_size,24},
                           {reductions,284}],
                          []]}}
[Sat, 14 Mar 2015 07:30:05 GMT] [error] [<0.88.0>] Unexpected exit of database process
<0.241.0> [<<"de-m-wikipedia-org">>]: {function_clause,
                                                                                         
    [{couch_compress,
                                                                                         
      decompress,
                                                                                         
      [<<>>]},
                                                                                         
     {couch_file,
                                                                                         
      pread_term,
                                                                                         
      2},
                                                                                         
     {couch_btree,
                                                                                         
      get_node,
                                                                                         
      2},
                                                                                         
     {couch_btree,
                                                                                         
      stream_node,
                                                                                         
      7},
                                                                                         
     {couch_btree,
                                                                                         
      stream_kp_node,
                                                                                         
      7},
                                                                                         
     {couch_btree,
                                                                                         
      stream_kp_node,
                                                                                         
      8},
                                                                                         
     {couch_btree,
                                                                                         
      fold,
                                                                                         
      4},
                                                                                         
     {couch_db_updater,
                                                                                         
      copy_compact,
                                                                                         
      3}]}



Mime
View raw message