couchdb-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Zhiqing Rao (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (COUCHDB-1856) hang and restart when replicate remotely of database that has doc > 10M, with 600kb/s network speed
Date Mon, 22 Jul 2013 09:38:48 GMT

     [ https://issues.apache.org/jira/browse/COUCHDB-1856?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Zhiqing Rao updated COUCHDB-1856:
---------------------------------

    Affects Version/s: 1.3.1
                       1.3

My system information for the target database:
------------------
System Information
------------------
Time of this report: 7/22/2013, 17:15:28
       Machine name: SHANGHAI-PERF1
   Operating System: Windows 7 Enterprise 64-bit (6.1, Build 7601) Service Pack 1 (7601.win7sp1_ldr.130318-1534)
           Language: English (Regional Setting: English)
System Manufacturer: Hewlett-Packard
       System Model: HP Z400 Workstation
               BIOS: Default System BIOS
          Processor: Intel(R) Xeon(R) CPU           W3550  @ 3.07GHz (4 CPUs), ~3.1GHz
             Memory: 6144MB RAM
Available OS Memory: 6128MB RAM
          Page File: 6669MB used, 5582MB available
        Windows Dir: C:\WINDOWS
    DirectX Version: DirectX 11
DX Setup Parameters: Not found
   User DPI Setting: Using System DPI
 System DPI Setting: 96 DPI (100 percent)
    DWM DPI Scaling: Disabled
     DxDiag Version: 6.01.7601.17514 32bit Unicode

And the couchdb.log
[Mon, 22 Jul 2013 08:57:02 GMT] [info] [<0.35.0>] Apache CouchDB has started on http://10.21.5.182:5984/
[Mon, 22 Jul 2013 08:57:25 GMT] [debug] [<0.147.0>] 'POST' /_replicate {1,1} from "10.253.9.174"
Headers: [{'Accept',"*/*"},
          {'Authorization',"Basic enJhbzpjb3VjaA=="},
          {'Content-Length',"143"},
          {'Content-Type',"application/json"},
          {'Host',"10.21.5.182:5984"},
          {'User-Agent',"curl/7.19.7 (i486-pc-linux-gnu) libcurl/7.19.7 OpenSSL/0.9.8k zlib/1.2.3.3
libidn/1.15"}]
[Mon, 22 Jul 2013 08:57:25 GMT] [debug] [<0.147.0>] OAuth Params: []
[Mon, 22 Jul 2013 08:57:26 GMT] [info] [<0.177.0>] Replication `"c225687064528107f5a4ba2b740edab2"`
is using:
	4 worker processes
	a worker batch size of 500
	10 HTTP connections
	a connection timeout of 30000 milliseconds
	10 retries per request
	socket options are: [{keepalive,true},{nodelay,false}]
	source start sequence 1558
[Mon, 22 Jul 2013 08:57:26 GMT] [debug] [<0.177.0>] Worker pids are: [<0.198.0>,<0.201.0>,<0.204.0>,<0.207.0>]
[Mon, 22 Jul 2013 08:57:26 GMT] [info] [<0.147.0>] starting new replication `c225687064528107f5a4ba2b740edab2`
at <0.177.0> (`http://mycouchdb:5984/mydb/` -> `mydb`)
[Mon, 22 Jul 2013 08:57:27 GMT] [debug] [<0.199.0>] Worker reported completion of seq
{1,1640}
[Mon, 22 Jul 2013 08:57:27 GMT] [debug] [<0.177.0>] Worker reported seq {1,1640}, through
seq was {0,1558}, new through seq is {1,
                                                                                         
      1640}, highest seq done was {0,
                                                                                         
                                   0}, new highest seq done is {1,
                                                                                         
                                                                1640}
Seqs in progress were: [{1,1640},{2,1670},{3,1672},{4,1673}]
Seqs in progress are now: [{2,1670},{3,1672},{4,1673}]
[Mon, 22 Jul 2013 08:57:28 GMT] [debug] [<0.202.0>] Worker reported completion of seq
{2,1670}
[Mon, 22 Jul 2013 08:57:28 GMT] [debug] [<0.177.0>] Worker reported seq {2,1670}, through
seq was {1,1640}, new through seq is {2,
                                                                                         
      1670}, highest seq done was {1,
                                                                                         
                                   1640}, new highest seq done is {2,
                                                                                         
                                                                   1670}
Seqs in progress were: [{2,1670},{3,1672},{4,1673}]
Seqs in progress are now: [{3,1672},{4,1673}]
[Mon, 22 Jul 2013 08:57:28 GMT] [debug] [<0.93.0>] New task status for <0.177.0>:
[{checkpointed_source_seq,
                                                    1558},
                                                   {continuous,false},
                                                   {doc_id,null},
                                                   {doc_write_failures,0},
                                                   {docs_read,0},
                                                   {docs_written,0},
                                                   {missing_revisions_found,0},
                                                   {progress,88},
                                                   {replication_id,
                                                    <<"c225687064528107f5a4ba2b740edab2">>},
                                                   {revisions_checked,1},
                                                   {source,
                                                    <<"http://mycouchdb:5984/mydb/">>},
                                                   {source_seq,1858},
                                                   {started_on,1374483446},
                                                   {target,<<"mydb">>},
                                                   {type,replication},
                                                   {updated_on,1374483448}]
[Mon, 22 Jul 2013 08:58:18 GMT] [debug] [<0.93.0>] New task status for <0.177.0>:
[{checkpointed_source_seq,
                                                    1558},
                                                   {continuous,false},
                                                   {doc_id,null},
                                                   {doc_write_failures,0},
                                                   {docs_read,0},
                                                   {docs_written,0},
                                                   {missing_revisions_found,0},
                                                   {progress,89},
                                                   {replication_id,
                                                    <<"c225687064528107f5a4ba2b740edab2">>},
                                                   {revisions_checked,2},
                                                   {source,
                                                    <<"http://mycouchdb:5984/mydb/">>},
                                                   {source_seq,1858},
                                                   {started_on,1374483446},
                                                   {target,<<"mydb">>},
                                                   {type,replication},
                                                   {updated_on,1374483498}]
[Mon, 22 Jul 2013 08:58:19 GMT] [info] [<0.177.0>] recording a checkpoint for `http://mycouchdb:5984/mydb/`
-> `mydb` at source update_seq 1670
[Mon, 22 Jul 2013 08:58:21 GMT] [debug] [<0.93.0>] New task status for <0.177.0>:
[{checkpointed_source_seq,
                                                    1670},
                                                   {continuous,false},
                                                   {doc_id,null},
                                                   {doc_write_failures,0},
                                                   {docs_read,0},
                                                   {docs_written,0},
                                                   {missing_revisions_found,0},
                                                   {progress,89},
                                                   {replication_id,
                                                    <<"c225687064528107f5a4ba2b740edab2">>},
                                                   {revisions_checked,2},
                                                   {source,
                                                    <<"http://mycouchdb:5984/mydb/">>},
                                                   {source_seq,1858},
                                                   {started_on,1374483446},
                                                   {target,<<"mydb">>},
                                                   {type,replication},
                                                   {updated_on,1374483501}]
[Mon, 22 Jul 2013 08:58:26 GMT] [debug] [<0.93.0>] New task status for <0.177.0>:
[{checkpointed_source_seq,
                                                    1670},
                                                   {continuous,false},
                                                   {doc_id,null},
                                                   {doc_write_failures,0},
                                                   {docs_read,0},
                                                   {docs_written,0},
                                                   {missing_revisions_found,0},
                                                   {progress,89},
                                                   {replication_id,
                                                    <<"c225687064528107f5a4ba2b740edab2">>},
                                                   {revisions_checked,2},
                                                   {source,
                                                    <<"http://mycouchdb:5984/mydb/">>},
                                                   {source_seq,1858},
                                                   {started_on,1374483446},
                                                   {target,<<"mydb">>},
                                                   {type,replication},
                                                   {updated_on,1374483506}]
[Mon, 22 Jul 2013 08:58:32 GMT] [debug] [<0.93.0>] New task status for <0.177.0>:
[{checkpointed_source_seq,
                                                    1670},
                                                   {continuous,false},
                                                   {doc_id,null},
                                                   {doc_write_failures,0},
                                                   {docs_read,0},
                                                   {docs_written,0},
                                                   {missing_revisions_found,0},
                                                   {progress,89},
                                                   {replication_id,
                                                    <<"c225687064528107f5a4ba2b740edab2">>},
                                                   {revisions_checked,2},
                                                   {source,
                                                    <<"http://mycouchdb:5984/mydb/">>},
                                                   {source_seq,1858},
                                                   {started_on,1374483446},
                                                   {target,<<"mydb">>},
                                                   {type,replication},
                                                   {updated_on,1374483512}]
[Mon, 22 Jul 2013 08:58:37 GMT] [debug] [<0.93.0>] New task status for <0.177.0>:
[{checkpointed_source_seq,
                                                    1670},
                                                   {continuous,false},
                                                   {doc_id,null},
                                                   {doc_write_failures,0},
                                                   {docs_read,0},
                                                   {docs_written,0},
                                                   {missing_revisions_found,0},
                                                   {progress,89},
                                                   {replication_id,
                                                    <<"c225687064528107f5a4ba2b740edab2">>},
                                                   {revisions_checked,2},
                                                   {source,
                                                    <<"http://mycouchdb:5984/mydb/">>},
                                                   {source_seq,1858},
                                                   {started_on,1374483446},
                                                   {target,<<"mydb">>},
                                                   {type,replication},
                                                   {updated_on,1374483517}]
[Mon, 22 Jul 2013 08:58:39 GMT] [debug] [<0.207.0>] Worker flushing doc batch of 1 docs
[Mon, 22 Jul 2013 08:58:40 GMT] [debug] [<0.208.0>] Worker reported completion of seq
{4,1673}
[Mon, 22 Jul 2013 08:58:40 GMT] [debug] [<0.177.0>] Worker reported seq {4,1673}, through
seq was {2,1670}, new through seq is {2,
                                                                                         
      1670}, highest seq done was {2,
                                                                                         
                                   1670}, new highest seq done is {4,
                                                                                         
                                                                   1673}
Seqs in progress were: [{3,1672},{4,1673},{5,1696},{6,1858}]
Seqs in progress are now: [{3,1672},{5,1696},{6,1858}]
[Mon, 22 Jul 2013 08:58:41 GMT] [debug] [<0.93.0>] New task status for <0.177.0>:
[{checkpointed_source_seq,
                                                    1670},
                                                   {continuous,false},
                                                   {doc_id,null},
                                                   {doc_write_failures,0},
                                                   {docs_read,1},
                                                   {docs_written,1},
                                                   {missing_revisions_found,1},
                                                   {progress,89},
                                                   {replication_id,
                                                    <<"c225687064528107f5a4ba2b740edab2">>},
                                                   {revisions_checked,3},
                                                   {source,
                                                    <<"http://mycouchdb:5984/mydb/">>},
                                                   {source_seq,1858},
                                                   {started_on,1374483446},
                                                   {target,<<"mydb">>},
                                                   {type,replication},
                                                   {updated_on,1374483521}]
[Mon, 22 Jul 2013 08:58:43 GMT] [debug] [<0.93.0>] New task status for <0.177.0>:
[{checkpointed_source_seq,
                                                    1670},
                                                   {continuous,false},
                                                   {doc_id,null},
                                                   {doc_write_failures,0},
                                                   {docs_read,1},
                                                   {docs_written,1},
                                                   {missing_revisions_found,1},
                                                   {progress,89},
                                                   {replication_id,
                                                    <<"c225687064528107f5a4ba2b740edab2">>},
                                                   {revisions_checked,3},
                                                   {source,
                                                    <<"http://mycouchdb:5984/mydb/">>},
                                                   {source_seq,1858},
                                                   {started_on,1374483446},
                                                   {target,<<"mydb">>},
                                                   {type,replication},
                                                   {updated_on,1374483523}]
[Mon, 22 Jul 2013 08:58:48 GMT] [debug] [<0.93.0>] New task status for <0.177.0>:
[{checkpointed_source_seq,
                                                    1670},
                                                   {continuous,false},
                                                   {doc_id,null},
                                                   {doc_write_failures,0},
                                                   {docs_read,1},
                                                   {docs_written,1},
                                                   {missing_revisions_found,1},
                                                   {progress,89},
                                                   {replication_id,
                                                    <<"c225687064528107f5a4ba2b740edab2">>},
                                                   {revisions_checked,3},
                                                   {source,
                                                    <<"http://mycouchdb:5984/mydb/">>},
                                                   {source_seq,1858},
                                                   {started_on,1374483446},
                                                   {target,<<"mydb">>},
                                                   {type,replication},
                                                   {updated_on,1374483528}]
[Mon, 22 Jul 2013 08:58:54 GMT] [debug] [<0.93.0>] New task status for <0.177.0>:
[{checkpointed_source_seq,
                                                    1670},
                                                   {continuous,false},
                                                   {doc_id,null},
                                                   {doc_write_failures,0},
                                                   {docs_read,1},
                                                   {docs_written,1},
                                                   {missing_revisions_found,1},
                                                   {progress,89},
                                                   {replication_id,
                                                    <<"c225687064528107f5a4ba2b740edab2">>},
                                                   {revisions_checked,3},
                                                   {source,
                                                    <<"http://mycouchdb:5984/mydb/">>},
                                                   {source_seq,1858},
                                                   {started_on,1374483446},
                                                   {target,<<"mydb">>},
                                                   {type,replication},
                                                   {updated_on,1374483534}]
[Mon, 22 Jul 2013 09:04:10 GMT] [info] [<0.35.0>] Apache CouchDB has started on http://10.21.5.182:5984/
[Mon, 22 Jul 2013 09:05:13 GMT] [debug] [<0.147.0>] 'GET' /_active_tasks {1,1} from
"10.253.9.174"
Headers: [{'Accept',"*/*"},
          {'Authorization',"Basic enJhbzpjb3VjaA=="},
          {'Host',"10.21.5.182:5984"},
          {'User-Agent',"curl/7.19.7 (i486-pc-linux-gnu) libcurl/7.19.7 OpenSSL/0.9.8k zlib/1.2.3.3
libidn/1.15"}]
[Mon, 22 Jul 2013 09:05:13 GMT] [debug] [<0.147.0>] OAuth Params: []
[Mon, 22 Jul 2013 09:05:13 GMT] [info] [<0.147.0>] 10.253.9.174 - - GET /_active_tasks
200

                
> hang and restart when replicate remotely of database that has doc > 10M, with 600kb/s
network speed
> ---------------------------------------------------------------------------------------------------
>
>                 Key: COUCHDB-1856
>                 URL: https://issues.apache.org/jira/browse/COUCHDB-1856
>             Project: CouchDB
>          Issue Type: Bug
>          Components: Replication
>    Affects Versions: 1.3, 1.3.1
>            Reporter: Zhiqing Rao
>
> When I remotely replicate a database that has doc > 10M, with 600kb/s network speed,
in a win7 64bit platform with couchdb 1.3.1, the couchdb server will launch the replication,
but erl.exe soon reach up to commiting > 2GB memory, then
> couchdb server hangs until a restart.
> Two things that might helpful:
> 1) It's fine for me to replicate the database in the same couchdb server (in the same
machine);  
> 2) My web browers, IE9/10, chrome, firefox, also hang or without response, at sometimes,
 when I open the documents in the database with URL; 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message