couchdb-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Maximillian Dornseif (JIRA)" <j...@apache.org>
Subject [jira] Commented: (COUCHDB-240) Replication breaks with large Attachments.
Date Mon, 09 Mar 2009 19:55:51 GMT

    [ https://issues.apache.org/jira/browse/COUCHDB-240?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12680249#action_12680249
] 

Maximillian Dornseif commented on COUCHDB-240:
----------------------------------------------

I'm now using r751801 on both machines.  I get a lot less error messages than before :-) Trying
"pull" replication I see on the destination machine:

[Mon, 09 Mar 2009 19:44:41 GMT] [error] [<0.2772.0>] streaming attachment failed with
connection_closed
[Mon, 09 Mar 2009 19:44:41 GMT] [error] [<0.65.0>] replicator-linked pid <0.2772.0>
exited with attachment_request_failed
[Mon, 09 Mar 2009 19:44:41 GMT] [error] [<0.2773.0>] streaming attachment failed with
connection_closed
[Mon, 09 Mar 2009 19:44:41 GMT] [error] [<0.65.0>] replication enumerator exited with
pls_restart_kthxbye .. respawning
[Mon, 09 Mar 2009 19:44:41 GMT] [error] [<0.65.0>] replicator-linked pid <0.2773.0>
exited with attachment_request_failed
[Mon, 09 Mar 2009 19:44:41 GMT] [error] [<0.65.0>] replication enumerator exited with
pls_restart_kthxbye .. respawning
[Mon, 09 Mar 2009 19:44:45 GMT] [error] [<0.2806.0>] streaming attachment failed with
connection_closed
[Mon, 09 Mar 2009 19:44:45 GMT] [error] [<0.65.0>] replicator-linked pid <0.2806.0>
exited with attachment_request_failed
[Mon, 09 Mar 2009 19:44:45 GMT] [error] [<0.65.0>] replication enumerator exited with
pls_restart_kthxbye .. respawning
[Mon, 09 Mar 2009 19:44:45 GMT] [error] [<0.2807.0>] streaming attachment failed with
connection_closed
[Mon, 09 Mar 2009 19:44:45 GMT] [error] [<0.65.0>] replicator-linked pid <0.2807.0>
exited with attachment_request_failed
[Mon, 09 Mar 2009 19:44:45 GMT] [error] [<0.65.0>] replication enumerator exited with
pls_restart_kthxbye .. respawning
... and so on.


On the source I see this:

[Mon, 09 Mar 2009 19:44:40 GMT] [info] [<0.516.0>] 192.168.0.30 - - 'GET' /huimages/MNGSB7ULVOYYVCXR01/11999_Snowboard_PRO_blau.jpg
200
[Mon, 09 Mar 2009 19:44:41 GMT] [info] [<0.466.0>] 192.168.0.30 - - 'GET' /huimages/AX5NEF3FGEIQQZHM01/IMG_6260.jpg
200
[Mon, 09 Mar 2009 19:44:41 GMT] [info] [<0.466.0>] 192.168.0.30 - - 'GET' /huimages/DSXPD7ZA4CR6TTFR01/74409.jpg
200
[Mon, 09 Mar 2009 19:44:41 GMT] [info] [<0.466.0>] 192.168.0.30 - - 'GET' /huimages/DSXPD7ZA4CR6TTFR01?revs=true&latest=true&open_revs=["124468297"]
200
[Mon, 09 Mar 2009 19:44:41 GMT] [info] [<0.466.0>] 192.168.0.30 - - 'GET' /huimages/66MUJ4H533ZXF3DL01/b36b0a51be757b6d21b338520c0439d5.jpg
200
[Mon, 09 Mar 2009 19:44:41 GMT] [info] [<0.466.0>] 192.168.0.30 - - 'GET' /huimages/66MUJ4H533ZXF3DL01?revs=true&latest=true&open_revs=["2626715626"]
200
[Mon, 09 Mar 2009 19:44:41 GMT] [info] [<0.506.0>] 192.168.0.30 - - 'GET' /huimages/FTNUGDTJDC4EX4NO01?revs=true&latest=true&open_revs=["2130525200"]
200
[Mon, 09 Mar 2009 19:44:41 GMT] [info] [<0.516.0>] 192.168.0.30 - - 'GET' /huimages/Z6EUBABSJJRFGGBW01?revs=true&latest=true&open_revs=["1174758156"]
200
[Mon, 09 Mar 2009 19:44:41 GMT] [info] [<0.523.0>] 192.168.0.30 - - 'GET' /huimages/53QLHO6TG5X4RUBO01?revs=true&latest=true&open_revs=["2043438337"]
200
[Mon, 09 Mar 2009 19:44:41 GMT] [info] [<0.523.0>] 192.168.0.30 - - 'GET' /huimages/J5XVD6QPSUC6QTYW01/New_01...JPG
200
[Mon, 09 Mar 2009 19:44:41 GMT] [info] [<0.523.0>] 192.168.0.30 - - 'GET' /huimages/J5XVD6QPSUC6QTYW01?revs=true&latest=true&open_revs=["4092060956"]
200
[Mon, 09 Mar 2009 19:44:41 GMT] [info] [<0.523.0>] 192.168.0.30 - - 'GET' /huimages/OJ5ZNVOKDCVFSSB701/76403.jpg
200
[Mon, 09 Mar 2009 19:44:41 GMT] [info] [<0.530.0>] 192.168.0.30 - - 'GET' /huimages/IP6MGMXSLIZSLT6S01?revs=true&latest=true&open_revs=["2604152110"]
200
[Mon, 09 Mar 2009 19:44:41 GMT] [info] [<0.530.0>] 192.168.0.30 - - 'GET' /huimages/XX75NZYJXOYY7BV501?revs=true&latest=true&open_revs=["3057582815"]
200
[Mon, 09 Mar 2009 19:44:41 GMT] [info] [<0.530.0>] 192.168.0.30 - - 'GET' /huimages/NAQO6H7RHLIOE7FM01?revs=true&latest=true&open_revs=["2756407557"]
200
[Mon, 09 Mar 2009 19:44:41 GMT] [info] [<0.558.0>] 192.168.0.30 - - 'GET' /huimages/YKKC3WWDZRD5IV7B01/57403.jpg
200
[Mon, 09 Mar 2009 19:44:41 GMT] [info] [<0.569.0>] 192.168.0.30 - - 'GET' /huimages/_all_docs_by_seq?limit=100&startkey=6880
200
[Mon, 09 Mar 2009 19:44:42 GMT] [info] [<0.523.0>] 192.168.0.30 - - 'GET' /huimages/53QLHO6TG5X4RUBO01/71570TR.jpg
200
[Mon, 09 Mar 2009 19:44:42 GMT] [info] [<0.523.0>] 192.168.0.30 - - 'GET' /huimages/Y6MEIUXGBOP5FUJI01/76597_.jpg
200
[Mon, 09 Mar 2009 19:44:42 GMT] [info] [<0.530.0>] 192.168.0.30 - - 'GET' /huimages/IP6MGMXSLIZSLT6S01/71576.jpg
200
[Mon, 09 Mar 2009 19:44:42 GMT] [info] [<0.530.0>] 192.168.0.30 - - 'GET' /huimages/5N52PUBPNXLOXF4J01/75008A.JPG
200
[Mon, 09 Mar 2009 19:44:42 GMT] [info] [<0.530.0>] 192.168.0.30 - - 'GET' /huimages/5N52PUBPNXLOXF4J01?revs=true&latest=true&open_revs=["3204386093"]
200
[Mon, 09 Mar 2009 19:44:42 GMT] [info] [<0.536.0>] 192.168.0.30 - - 'GET' /huimages/AX5NEF3FGEIQQZHM01?revs=true&latest=true&open_revs=["4285217492"]
200
[Mon, 09 Mar 2009 19:44:42 GMT] [info] [<0.536.0>] 192.168.0.30 - - 'GET' /huimages/J2XORR3HRCSMXF7701/65294.JPG
200
[Mon, 09 Mar 2009 19:44:42 GMT] [info] [<0.536.0>] 192.168.0.30 - - 'GET' /huimages/J2XORR3HRCSMXF7701?revs=true&latest=true&open_revs=["3119947216"]
200
[Mon, 09 Mar 2009 19:44:42 GMT] [info] [<0.550.0>] 192.168.0.30 - - 'GET' /huimages/M3G6MZBIYJRYWZU301/Multitisch_2006_.jpg
200
[Mon, 09 Mar 2009 19:44:42 GMT] [info] [<0.550.0>] 192.168.0.30 - - 'GET' /huimages/M3G6MZBIYJRYWZU301?revs=true&latest=true&open_revs=["166021924"]
200
[Mon, 09 Mar 2009 19:44:42 GMT] [info] [<0.552.0>] 192.168.0.30 - - 'GET' /huimages/M2CFHVLEWQORIMCB01/Kicker_ST_1018.jpg
200
[Mon, 09 Mar 2009 19:44:42 GMT] [info] [<0.552.0>] 192.168.0.30 - - 'GET' /huimages/M2CFHVLEWQORIMCB01?revs=true&latest=true&open_revs=["3664679569"]
200
[Mon, 09 Mar 2009 19:44:42 GMT] [info] [<0.558.0>] 192.168.0.30 - - 'GET' /huimages/Y6MEIUXGBOP5FUJI01?revs=true&latest=true&open_revs=["3391344911"]
200
[Mon, 09 Mar 2009 19:44:43 GMT] [info] [<0.506.0>] 192.168.0.30 - - 'GET' /huimages/FTNUGDTJDC4EX4NO01/71422.jpg
200
[Mon, 09 Mar 2009 19:44:43 GMT] [info] [<0.523.0>] 192.168.0.30 - - 'GET' /huimages/F2RRFOVPZFHQLR4301/74404.jpg
200
[Mon, 09 Mar 2009 19:44:43 GMT] [info] [<0.523.0>] 192.168.0.30 - - 'GET' /huimages/AX5NEF3FGEIQQZHM01/IMG_6260.jpg
200
[Mon, 09 Mar 2009 19:44:43 GMT] [info] [<0.530.0>] 192.168.0.30 - - 'GET' /huimages/YDFKQ45LC3MDHSOD01/75210_Sportlenkdrachen_LD-120_KS_3049.png
200
[Mon, 09 Mar 2009 19:44:43 GMT] [info] [<0.530.0>] 192.168.0.30 - - 'GET' /huimages/YDFKQ45LC3MDHSOD01?revs=true&latest=true&open_revs=["2136613132"]
200
[Mon, 09 Mar 2009 19:44:43 GMT] [info] [<0.536.0>] 192.168.0.30 - - 'GET' /huimages/L2Z6TGHZBNKIP6TS01/Neopren_weights_Lady_green.jpg
200
[Mon, 09 Mar 2009 19:44:43 GMT] [info] [<0.536.0>] 192.168.0.30 - - 'GET' /huimages/L2Z6TGHZBNKIP6TS01?revs=true&latest=true&open_revs=["1847492796"]
200
[Mon, 09 Mar 2009 19:44:43 GMT] [info] [<0.536.0>] 192.168.0.30 - - 'GET' /huimages/MNGSB7ULVOYYVCXR01?revs=true&latest=true&open_revs=["1794236561"]
200
[Mon, 09 Mar 2009 19:44:43 GMT] [info] [<0.550.0>] 192.168.0.30 - - 'GET' /huimages/M2C4FP754G2QNJ2K01/Neopren_weights_Lady_purple.jpg
200
[Mon, 09 Mar 2009 19:44:43 GMT] [info] [<0.550.0>] 192.168.0.30 - - 'GET' /huimages/M2C4FP754G2QNJ2K01?revs=true&latest=true&open_revs=["1618418127"]
200
[Mon, 09 Mar 2009 19:44:43 GMT] [info] [<0.550.0>] 192.168.0.30 - - 'GET' /huimages/OJ5ZNVOKDCVFSSB701/76403.jpg
200
[Mon, 09 Mar 2009 19:44:43 GMT] [info] [<0.552.0>] 192.168.0.30 - - 'GET' /huimages/A5Z6E5ZTBSXFIAVC01/72760.jpg
200
[Mon, 09 Mar 2009 19:44:43 GMT] [info] [<0.552.0>] 192.168.0.30 - - 'GET' /huimages/A5Z6E5ZTBSXFIAVC01?revs=true&latest=true&open_revs=["1263711090"]
200
[Mon, 09 Mar 2009 19:44:43 GMT] [info] [<0.552.0>] 192.168.0.30 - - 'GET' /huimages/WXW2AJYKXDQ2YY6Z01?revs=true&latest=true&open_revs=["628231131"]
200
[Mon, 09 Mar 2009 19:44:43 GMT] [info] [<0.558.0>] 192.168.0.30 - - 'GET' /huimages/F2RRFOVPZFHQLR4301?revs=true&latest=true&open_revs=["284397338"]
200
[Mon, 09 Mar 2009 19:44:43 GMT] [info] [<0.558.0>] 192.168.0.30 - - 'GET' /huimages/OJ5ZNVOKDCVFSSB701?revs=true&latest=true&open_revs=["471519366"]
200
[Mon, 09 Mar 2009 19:44:44 GMT] [info] [<0.466.0>] 192.168.0.30 - - 'GET' /huimages/XX75NZYJXOYY7BV501/71590TR.jpg
200
[Mon, 09 Mar 2009 19:44:44 GMT] [info] [<0.466.0>] 192.168.0.30 - - 'GET' /huimages/WXW2AJYKXDQ2YY6Z01/74406_Kopie.jpg
200
[Mon, 09 Mar 2009 19:44:44 GMT] [info] [<0.506.0>] 192.168.0.30 - - 'GET' /huimages/2QZPNUA3IGOPKUAT01?revs=true&latest=true&open_revs=["1401971826"]
200
[Mon, 09 Mar 2009 19:44:44 GMT] [info] [<0.523.0>] 192.168.0.30 - - 'GET' /huimages/MUQ7IUD37UBIYBHQ01/57104__.jpg
200
[Mon, 09 Mar 2009 19:44:44 GMT] [info] [<0.523.0>] 192.168.0.30 - - 'GET' /huimages/MUQ7IUD37UBIYBHQ01?revs=true&latest=true&open_revs=["2964667508"]
200
[Mon, 09 Mar 2009 19:44:44 GMT] [info] [<0.550.0>] 192.168.0.30 - - 'GET' /huimages/YKKC3WWDZRD5IV7B01/57403.jpg
200
[Mon, 09 Mar 2009 19:44:44 GMT] [info] [<0.550.0>] 192.168.0.30 - - 'GET' /huimages/YKKC3WWDZRD5IV7B01?revs=true&latest=true&open_revs=["2242812618"]
200
[Mon, 09 Mar 2009 19:44:44 GMT] [info] [<0.552.0>] 192.168.0.30 - - 'GET' /huimages/UVJV6ONQL22R4T7Z01?revs=true&latest=true&open_revs=["3010604255"]
200
[Mon, 09 Mar 2009 19:44:45 GMT] [info] [<0.466.0>] 192.168.0.30 - - 'GET' /huimages/UVJV6ONQL22R4T7Z01/74401.jpg
200
[Mon, 09 Mar 2009 19:44:45 GMT] [info] [<0.506.0>] 192.168.0.30 - - 'GET' /huimages/2QZPNUA3IGOPKUAT01/74407.jpg
200
[Mon, 09 Mar 2009 19:44:45 GMT] [info] [<0.523.0>] 192.168.0.30 - - 'GET' /huimages/DSXPD7ZA4CR6TTFR01/74409.jpg
200
[Mon, 09 Mar 2009 19:44:45 GMT] [info] [<0.523.0>] 192.168.0.30 - - 'GET' /huimages/DSXPD7ZA4CR6TTFR01?revs=true&latest=true&open_revs=["124468297"]
200
[Mon, 09 Mar 2009 19:44:45 GMT] [info] [<0.523.0>] 192.168.0.30 - - 'GET' /huimages/66MUJ4H533ZXF3DL01/b36b0a51be757b6d21b338520c0439d5.jpg
200
[Mon, 09 Mar 2009 19:44:45 GMT] [info] [<0.523.0>] 192.168.0.30 - - 'GET' /huimages/66MUJ4H533ZXF3DL01?revs=true&latest=true&open_revs=["2626715626"]
200
[Mon, 09 Mar 2009 19:44:45 GMT] [info] [<0.523.0>] 192.168.0.30 - - 'GET' /huimages/_all_docs_by_seq?limit=100&startkey=6880
200
[Mon, 09 Mar 2009 19:44:45 GMT] [info] [<0.530.0>] 192.168.0.30 - - 'GET' /huimages/MNGSB7ULVOYYVCXR01/11999_Snowboard_PRO_blau.jpg
200
[Mon, 09 Mar 2009 19:44:45 GMT] [info] [<0.536.0>] 192.168.0.30 - - 'GET' /huimages/FTNUGDTJDC4EX4NO01/71422.jpg
200
[Mon, 09 Mar 2009 19:44:45 GMT] [info] [<0.536.0>] 192.168.0.30 - - 'GET' /huimages/FTNUGDTJDC4EX4NO01?revs=true&latest=true&open_revs=["2130525200"]
200
[Mon, 09 Mar 2009 19:44:45 GMT] [info] [<0.550.0>] 192.168.0.30 - - 'GET' /huimages/Z6EUBABSJJRFGGBW01?revs=true&latest=true&open_revs=["1174758156"]
200
[Mon, 09 Mar 2009 19:44:45 GMT] [info] [<0.550.0>] 192.168.0.30 - - 'GET' /huimages/J5XVD6QPSUC6QTYW01?revs=true&latest=true&open_revs=["4092060956"]
200
[Mon, 09 Mar 2009 19:44:45 GMT] [info] [<0.550.0>] 192.168.0.30 - - 'GET' /huimages/NAQO6H7RHLIOE7FM01?revs=true&latest=true&open_revs=["2756407557"]
200
[Mon, 09 Mar 2009 19:44:45 GMT] [info] [<0.576.0>] 192.168.0.30 - - 'GET' /huimages/_all_docs_by_seq?limit=100&startkey=6880
200
[Mon, 09 Mar 2009 19:44:46 GMT] [info] [<0.523.0>] 192.168.0.30 - - 'GET' /huimages/J5XVD6QPSUC6QTYW01/New_01...JPG
200


I see no exceptions anywhere. On the futon status screen I see "Processed source update #6880".
The number starts lower but always stops updating at #6880.

Adam, if you want to, I can make the DB in question (31GB) available to you.

> Replication breaks with large Attachments.
> ------------------------------------------
>
>                 Key: COUCHDB-240
>                 URL: https://issues.apache.org/jira/browse/COUCHDB-240
>             Project: CouchDB
>          Issue Type: Bug
>          Components: Database Core
>    Affects Versions: 0.9
>         Environment: r 741265. Debian Linux unknown revision, FreeBSD 7.0. GBit Network
connection between the hosts.
>            Reporter: Maximillian Dornseif
>
> I use the code in http://code.google.com/p/couchdb-python/issues/detail?id=54 to do replication
between two machines.
> I'm running 741265 on both machines. I have a Database with big attachments (high-res
images, 31.1 GB,	34026 Docs). "Pull" replication breaks with following message sent via http:
> couchdb.client.ServerError: (500, ('function_clause', "[{lists,map,[#Fun<couch_rep.10.28922857>,ok]},\n
{couch_rep,open_doc_revs,4},\n {couch_rep,'-enum_docs_parallel/3-fun-1-',3},\n {couch_rep,'-spawn_worker/3-fun-0-',3}]"))
> With "push" replication the server just drops the connection  (httplib2/__init__.py",
line 715, in connect
> socket.error: (61, 'Connection refused') - why "refused" instead of "closed"?). I have
only been able to replicate the first 100 documents.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message