Mailing-List: contact dev-help@couchdb.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@couchdb.apache.org
Date: Thu, 15 Sep 2016 06:23:57 -0400 (EDT)
From: Joan Touzet <wohali@apache.org>
Reply-To: Joan Touzet <wohali@apache.org>
To: dev@couchdb.apache.org
Message-ID: <1387326.1412.1473935144913.JavaMail.Joan@RITA>
In-Reply-To: <BN4PR10MB0995D26B5AF047D8386F2A0CA2F10@BN4PR10MB0995.namprd10.prod.outlook.com>
References: <BAY174-W27E4B81B76BF4D2132E13BA2FA0@phx.gbl> <BAY174-W25B0C3C1ABF4BAD99B8937A2FA0@phx.gbl> <2883838.505.1473451976224.JavaMail.Joan@RITA> <BN4PR10MB0995EC1FA0BF1E268EBD2C2BA2F10@BN4PR10MB0995.namprd10.prod.outlook.com> <25200628.1118.1473824952732.JavaMail.Joan@RITA> <BN4PR10MB0995D1EA62B6732AD0008E05A2F10@BN4PR10MB0995.namprd10.prod.outlook.com> <27966981.1153.1473829264061.JavaMail.Joan@RITA> <BN4PR10MB0995D26B5AF047D8386F2A0CA2F10@BN4PR10MB0995.namprd10.prod.outlook.com>
Subject: Re: High CPU usage caused by clustering and/or replication?
MIME-Version: 1.0
Content-Type: multipart/alternative;
	boundary="----=_Part_1411_5650592.1473935144909"
Thread-Topic: High CPU usage caused by clustering and/or replication?
Thread-Index: AQHSDYgcPrswn7modEmrLcCn02LcZKBxkY4AgAa5dG6Tnk4fsOxhwqeLPTxXrBP+Fo+ctY7rlHWU
archived-at: Thu, 15 Sep 2016 10:26:10 -0000

------=_Part_1411_5650592.1473935144909
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit

Best I can suggest at this point is to send on your logfiles, 
and if you have any data that helps reproduce the problem you 
are experiencing that you can share, that'd be great, too. 

If you can provide this data, please file a new ticket at 
https://issues.apache.org/jira/secure/CreateIssue!default.jspa 
selecting CouchDB as the project. 

-Joan 

----- Original Message -----

> From: "Joey Samonte" <csharpdeveloper@hotmail.com>
> To: dev@couchdb.apache.org, "Joan Touzet" <wohali@apache.org>
> Sent: Wednesday, September 14, 2016 6:59:12 AM
> Subject: Re: High CPU usage caused by clustering and/or replication?

> Same thing is happening on a Linux cluster. :( It even causes the
> beam process to crash.
> I am using free tier instances on AWS, with 1 Core and 1GB RAM. Would
> that cause this issue?

> From: Joan Touzet <wohali@apache.org>
> Sent: Wednesday, September 14, 2016 4:59 AM
> To: dev@couchdb.apache.org
> Subject: Re: High CPU usage caused by clustering and/or replication?

> I would recommend that you try and reproduce the scenario you're
> running into with a Linux-based cluster, and if you find it there
> as well, file a bug with us so we can try and work your problem out.

> If the problem goes away, then yes, I recommend for now you consider
> a
> Linux cluster. ;)

> I also recommend you use a minimum of 3 nodes for a cluster of q=8
> (the
> default).

> ----- Original Message -----
> > From: "Joey Samonte" <csharpdeveloper@hotmail.com>
> > To: dev@couchdb.apache.org, "Joan Touzet" <wohali@apache.org>
> > Sent: Wednesday, September 14, 2016 12:04:04 AM
> > Subject: Re: High CPU usage caused by clustering and/or
> > replication?
> >
> > I actually stopped all replications and the high CPU occurs on the
> > 2nd node. I have only 2 nodes in the cluster.
> >
> > We really want to move to CouchDB 2.0, but we can't afford this
> > issue
> > I am encountering.
> >
> > Do you suggest we just use Linux-based cluster? :(
> >
> >
> >
> > ________________________________
> > From: Joan Touzet <wohali@apache.org>
> > Sent: Wednesday, September 14, 2016 3:47 AM
> > To: dev@couchdb.apache.org
> > Subject: Re: High CPU usage caused by clustering and/or
> > replication?
> >
> > Joey,
> >
> > I'm sorry, I can't reproduce this problem. If you're able to
> > provide
> > sample data that causes this issue, I'd be pleased to test (on a
> > non-virtualized Windows machine) with that.
> >
> > I will note that I am only testing with a local single CouchDB
> > server,
> > not in a cluster. What if you try replicating to a non-clustered
> > Windows server? We have absolutely zero testing done on CouchDB
> > clustered on Windows, and in fact really should declare our support
> > for
> > that configuration as beta. I will state I have not set up a local
> > Windows-based cluster (on separate Windows machines; I have run a 3
> > node-on-1-server development cluster setup) so I can't guarantee
> > that
> > this isn't the problem.
> >
> > Please note this Known Issue in CouchDB 2.0:
> > https://issues.apache.org/jira/browse/COUCHDB-2980
> >
> > To quote our Known Issues document: "for now, we recommend always
> > using
> > the URL format for both source and target specifications."
> >
> > Finally, you should know it was my decision to continue Windows
> > support
> > for CouchDB, as no one else was able to step up. I did so primarily
> > because, in my experience, we have developers who develop against a
> > local CouchDB instance on their Windows laptops, then move to a
> > Linux-hosted cluster for server use. I didn't expect people would
> > be
> > trying to leverage the Windows port in a real server configuration.
> >
> > That's not to say we couldn't eventually support Windows in a
> > server
> > cluster configuration as "out of beta," but it'll take more
> > experience
> > and more repeatable bugs at the very least - and active CouchDB
> > Windows
> > developers with performance tuning experience at best.
> >
> > -Joan
> >
> > ----- Original Message -----
> > > From: "Joey Samonte" <csharpdeveloper@hotmail.com>
> > > To: dev@couchdb.apache.org, "Joan Touzet" <wohali@apache.org>
> > > Sent: Tuesday, September 13, 2016 10:53:18 PM
> > > Subject: Re: High CPU usage caused by clustering and/or
> > > replication?
> > >
> > > I have tried to stop all replications, then restarted CouchDB,
> > > but
> > > still having almost 100% CPU with erl.exe
> > >
> > >
> > > ________________________________
> > > From: Joan Touzet <wohali@apache.org>
> > > Sent: Friday, September 9, 2016 8:11 PM
> > > To: dev@couchdb.apache.org
> > > Subject: Re: High CPU usage caused by clustering and/or
> > > replication?
> > >
> > > I don't know, a simple view with 100 documents should build
> > > quickly
> > > unless your view code is unusual.
> > >
> > > I would recommend replicating the database to a UNIX-based 2.0
> > > cluster
> > > to see if the problem is Windows-specific.
> > >
> > > Sorry that I don't have any other ideas at the moment.
> > >
> > > -Joan
> > >
> > > ----- Original Message -----
> > > > From: "Joey Samonte" <csharpdeveloper@hotmail.com>
> > > > To: dev@couchdb.apache.org
> > > > Sent: Friday, September 9, 2016 3:08:37 PM
> > > > Subject: RE: High CPU usage caused by clustering and/or
> > > > replication?
> > > >
> > > > Hi ma'am Joan,
> > > >
> > > > But if the cause is a view being rebuilt, why is it so high? I
> > > > only
> > > > have less than 100 documents on that database which I wanted to
> > > > test
> > > > initially.
> > > >
> > > > > Date: Fri, 9 Sep 2016 07:10:35 -0400
> > > > > From: wohali@apache.org
> > > > > To: dev@couchdb.apache.org
> > > > > Subject: Re: High CPU usage caused by clustering and/or
> > > > > replication?
> > > > >
> > > > > Hi Joey,
> > > > >
> > > > > I'm going to oversimplify a bit here to make the explanation
> > > > > easier.
> > > > >
> > > > > If you check the process manager/top on each node, do you see
> > > > > couchjs
> > > > > responsible for that 100% CPU?
> > > > >
> > > > > Remember that views are not replicated, only documents. So if
> > > > > you
> > > > > have
> > > > > design documents that you replicated, those views may be
> > > > > being
> > > > > rebuilt.
> > > > >
> > > > > So-called "internal replication" will be placing segments of
> > > > > the
> > > > > database
> > > > > onto each node, and view building will be happening on each
> > > > > node
> > > > > as
> > > > > well.
> > > > > Since the view is also sharded, each shard of the database
> > > > > gets
> > > > > its
> > > > > own
> > > > > view shard; that has to be calculated independently on each
> > > > > node.
> > > > >
> > > > > Even if you're not building views, remember that a cluster
> > > > > places
> > > > > part
> > > > > of the database on each node, spreading the load out as
> > > > > equally
> > > > > as
> > > > > possible. In a standard 3-node cluster, each document write
> > > > > will
> > > > > be
> > > > > written redundantly to every node, meaning CPU load will be
> > > > > (mostly)
> > > > > equal across all 3 nodes as you write that document.
> > > > >
> > > > > Hope this helps,
> > > > > Joan
> > > > >
> > > > >
> > > > > ----- Original Message -----
> > > > > > From: "Joey Samonte" <csharpdeveloper@hotmail.com>
> > > > > > To: dev@couchdb.apache.org
> > > > > > Sent: Thursday, September 8, 2016 11:30:46 PM
> > > > > > Subject: High CPU usage caused by clustering and/or
> > > > > > replication?
> > > > > >
> > > > > > Good day,
> > > > > >
> > > > > > I have setup a cluster with two nodes, and I am replicating
> > > > > > the
> > > > > > first
> > > > > > node to an existing database through a loadbalancer. But I
> > > > > > am
> > > > > > getting a 100% CPU usage after a few minutes on both nodes.
> > > > > > Is
> > > > > > this
> > > > > > an issue with RC4?
> > > > > >
> > > > > > Thanks.
> > > > > >
> > > >
> > >
> >

------=_Part_1411_5650592.1473935144909--