couchdb-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Scott Weber <scotty2...@sbcglobal.net>
Subject Re: Replication of attachment is extremely slow
Date Fri, 24 Jan 2014 15:29:05 GMT
Well, as much as I hate to be a victim of "operator error", it appears that is the case. 
There is still an issue, but it seems to be dependent on an external process.

What I did:
Stopped the service, and removed the logs and 'to'/'from' databases.  Started the service,
and uploaded a 4 meg file as an attachment.  Then triggered a replication.  It took over
30 seconds.  I repeated this test, and it was very long to replicate every time.

Figuring I would post the log and the 4 meg file, I looked in the log for confidential stuff. 
I then went back and disabled some script daemons running from settings in the INI, as well
as the continuous replications in the _replicator database.  The PHP scripts are largely
idle, but trigger a query command every 30 seconds or so.

When I repeated the test, it was instantaneous.  Several times.  I could get a long delay
result.

So it would appear (at least so far) that some PHP script was causing a thread or socket to
block.  At this stage I can't tell which side the blame is on, but I can speculate that it
is something going on between the two of them.

So, steering the conversation another way, is there a potential problem with with perhaps
leaving an open socket connection after a replication command has been issued?

I will start to re-enable these other components and see when the issue returns.

-Scott





----- Original Message -----
From: Nick North <north.n@gmail.com>
To: "user@couchdb.apache.org" <user@couchdb.apache.org>
Cc: 
Sent: Friday, January 24, 2014 7:06 AM
Subject: Re: Replication of attachment is extremely slow

On 24 January 2014 12:44, Dave Cottlehuber <dch@jsonified.com> wrote:

> On 24. Jänner 2014 at 08:23:16, Nick North (north.n@gmail.com) wrote:
> > From: Nick North <north.n@gmail.com>
>
> > There is a known inefficiency
> > in parsing of attachments containing certain character strings.
> > This doesn't obviously sound like an instance of it, but you can
> > eliminate the possibility very quickly if you replace all hyphens
> > (i.e. the character "-") in your mixed content file by some other
> > character, like "X". If replication of the modified file runs
> > quickly, you've hit this particular problem.
> > Nick
>
> Hey Nick, https://issues.apache.org/jira/browse/COUCHDB-1953 thanks
> for bringing that up. I’ve quickly re-done my timing checks with that
> and see no major difference, do you want to compare a few with some
> other modified test docs and see how that looks?
>
> A+
> Dave
>
>
I'm not really expecting this problem to be the cause of the slowdown:
the attachment needs to contain a lot of initial prefixes of the MIME
boundary string for things to be really bad. While the patch speeds up
parsing in all cases, I suspect the processing time is usually dominated by
other parts of the process. I just mentioned it as it's the only bit of
code I know whose performance varies with attachment content, so it's worth
a try. It would be great if Scott is able to put his offending file up
somewhere for us to try out.

Nick


Mime
View raw message