couchdb-replication mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Rian R. Maloney" <rian.malo...@yahoo.com>
Subject Re: Replication of attachment is extremely slow.. LOGGED INFORMATION
Date Fri, 24 Jan 2014 21:03:58 GMT
Thanks EVERYONE for the assistance on this one. Much appreciated! Great user group

The following curl batch script consistently recreates the problem. 

The difference between a fast and slow replication is whether or not the full URL is used
for the "from" and "to" databases

cls
echo off

echo start from scratch
c:\es\curl\curl -X DELETE http://admin:admin@127.0.0.1:5984/from
c:\es\curl\curl -X DELETE http://admin:admin@127.0.0.1:5984/to
rem c:\es\curl\curl -X DELETE http://admin:admin@127.0.0.1:5984/_replicator/test?rev=

return

echo create from database
c:\es\curl\curl -X PUT http://admin:admin@127.0.0.1:5984/from

echo _
echo create to database
c:\es\curl\curl -X PUT http://admin:admin@127.0.0.1:5984/to

echo _
echo create dummy document to hold attachment
c:\es\curl\curl -X PUT -d "{\"dummy\":\"slow\"}" http://admin:admin@127.0.0.1:5984/from/test

echo _
echo upload the problem attachment - BE SURE TO SET THE REVISION NUMBER 
c:\es\curl\curl -X PUT http://127.0.0.1:5984/from/test/x937?rev=1-23a888c74ba58fe20dc0d424133750bb
--data-binary @slow.x937 -H "Content-Type: application/octet-stream"

echo _
echo time before %TIME%
echo _

rem echo FAST REPLICATION %TIME%
rem c:\es\curl\curl -H "Content-Type: application/json" -X POST -d "{\"source\":\"from\",\"target\":\"to\"}"
http://admin:admin@127.0.0.1:5984/_replicate


echo SLOW REPLICATION %TIME%
c:\es\curl\curl -H "Content-Type: application/json" -X POST -d "{\"source\":\"http://127.0.0.1:5984/from\",\"target\":\"http://127.0.0.1:5984/to\"}"
http://admin:admin@127.0.0.1:5984/_replicate


echo time after  %TIME%
 



On Friday, January 24, 2014 2:26 PM, Paul Davis <paul.joseph.davis@gmail.com> wrote:
 
The important thing to note here is that test 1 is using URLs for
databases where tests 2 and 3 are using internal replication. Reading
the logs it shows that the request to write the attachment took about
72 seconds. Out of curiosity could you rerun test 3 twice more with
the following JSON bodies:

{"from":"http://localhost:5984/from", "to":"to"}
{"from":"from", "to":"http://localhost:5984/to"}

This should narrow down if there's a problem with reading attachments,
writing attachments, or some weird interplay between reading and
writing attachments.

If it turns out that the only slow version is when both are using the
"http://localhost:5984/dbname" versions then can you run one more
test:

{"from":"http://127.0.0.1:5984/from", "to":"http://localhost:5984/to"}

If that's also slow then we probably have a weird problem with the
multipart code (fun!). If its not slow then most likely this is a bug
in the HTTP worker pool implementation (also fun!).
Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message