couchdb-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Joey Samonte <csharpdevelo...@hotmail.com>
Subject Re: High CPU usage caused by clustering and/or replication?
Date Wed, 14 Sep 2016 10:59:12 GMT
Same thing is happening on a Linux cluster. :( It even causes the beam process to crash.

I am using free tier instances on AWS, with 1 Core and 1GB RAM. Would that cause this issue?



________________________________
From: Joan Touzet <wohali@apache.org>
Sent: Wednesday, September 14, 2016 4:59 AM
To: dev@couchdb.apache.org
Subject: Re: High CPU usage caused by clustering and/or replication?

I would recommend that you try and reproduce the scenario you're
running into with a Linux-based cluster, and if you find it there
as well, file a bug with us so we can try and work your problem out.

If the problem goes away, then yes, I recommend for now you consider a
Linux cluster. ;)

I also recommend you use a minimum of 3 nodes for a cluster of q=8 (the
default).

----- Original Message -----
> From: "Joey Samonte" <csharpdeveloper@hotmail.com>
> To: dev@couchdb.apache.org, "Joan Touzet" <wohali@apache.org>
> Sent: Wednesday, September 14, 2016 12:04:04 AM
> Subject: Re: High CPU usage caused by clustering and/or replication?
>
> I actually stopped all replications and the high CPU occurs on the
> 2nd node. I have only 2 nodes in the cluster.
>
> We really want to move to CouchDB 2.0, but we can't afford this issue
> I am encountering.
>
> Do you suggest we just use Linux-based cluster? :(
>
>
>
> ________________________________
> From: Joan Touzet <wohali@apache.org>
> Sent: Wednesday, September 14, 2016 3:47 AM
> To: dev@couchdb.apache.org
> Subject: Re: High CPU usage caused by clustering and/or replication?
>
> Joey,
>
> I'm sorry, I can't reproduce this problem. If you're able to provide
> sample data that causes this issue, I'd be pleased to test (on a
> non-virtualized Windows machine) with that.
>
> I will note that I am only testing with a local single CouchDB
> server,
> not in a cluster.  What if you try replicating to a non-clustered
> Windows server? We have absolutely zero testing done on CouchDB
> clustered on Windows, and in fact really should declare our support
> for
> that configuration as beta. I will state I have not set up a local
> Windows-based cluster (on separate Windows machines; I have run a 3
> node-on-1-server development cluster setup) so I can't guarantee that
> this isn't the problem.
>
> Please note this Known Issue in CouchDB 2.0:
>   https://issues.apache.org/jira/browse/COUCHDB-2980
>
> To quote our Known Issues document: "for now, we recommend always
> using
> the URL format for both source and target specifications."
>
> Finally, you should know it was my decision to continue Windows
> support
> for CouchDB, as no one else was able to step up. I did so primarily
> because, in my experience, we have developers who develop against a
> local CouchDB instance on their Windows laptops, then move to a
> Linux-hosted cluster for server use. I didn't expect people would be
> trying to leverage the Windows port in a real server configuration.
>
> That's not to say we couldn't eventually support Windows in a server
> cluster configuration as "out of beta," but it'll take more
> experience
> and more repeatable bugs at the very least - and active CouchDB
> Windows
> developers with performance tuning experience at best.
>
> -Joan
>
> ----- Original Message -----
> > From: "Joey Samonte" <csharpdeveloper@hotmail.com>
> > To: dev@couchdb.apache.org, "Joan Touzet" <wohali@apache.org>
> > Sent: Tuesday, September 13, 2016 10:53:18 PM
> > Subject: Re: High CPU usage caused by clustering and/or
> > replication?
> >
> > I have tried to stop all replications, then restarted CouchDB, but
> > still having almost 100% CPU with erl.exe
> >
> >
> > ________________________________
> > From: Joan Touzet <wohali@apache.org>
> > Sent: Friday, September 9, 2016 8:11 PM
> > To: dev@couchdb.apache.org
> > Subject: Re: High CPU usage caused by clustering and/or
> > replication?
> >
> > I don't know, a simple view with 100 documents should build quickly
> > unless your view code is unusual.
> >
> > I would recommend replicating the database to a UNIX-based 2.0
> > cluster
> > to see if the problem is Windows-specific.
> >
> > Sorry that I don't have any other ideas at the moment.
> >
> > -Joan
> >
> > ----- Original Message -----
> > > From: "Joey Samonte" <csharpdeveloper@hotmail.com>
> > > To: dev@couchdb.apache.org
> > > Sent: Friday, September 9, 2016 3:08:37 PM
> > > Subject: RE: High CPU usage caused by clustering and/or
> > > replication?
> > >
> > > Hi ma'am Joan,
> > >
> > > But if the cause is a view being rebuilt, why is it so high? I
> > > only
> > > have less than 100 documents on that database which I wanted to
> > > test
> > > initially.
> > >
> > > > Date: Fri, 9 Sep 2016 07:10:35 -0400
> > > > From: wohali@apache.org
> > > > To: dev@couchdb.apache.org
> > > > Subject: Re: High CPU usage caused by clustering and/or
> > > > replication?
> > > >
> > > > Hi Joey,
> > > >
> > > > I'm going to oversimplify a bit here to make the explanation
> > > > easier.
> > > >
> > > > If you check the process manager/top on each node, do you see
> > > > couchjs
> > > > responsible for that 100% CPU?
> > > >
> > > > Remember that views are not replicated, only documents. So if
> > > > you
> > > > have
> > > > design documents that you replicated, those views may be being
> > > > rebuilt.
> > > >
> > > > So-called "internal replication" will be placing segments of
> > > > the
> > > > database
> > > > onto each node, and view building will be happening on each
> > > > node
> > > > as
> > > > well.
> > > > Since the view is also sharded, each shard of the database gets
> > > > its
> > > > own
> > > > view shard; that has to be calculated independently on each
> > > > node.
> > > >
> > > > Even if you're not building views, remember that a cluster
> > > > places
> > > > part
> > > > of the database on each node, spreading the load out as equally
> > > > as
> > > > possible. In a standard 3-node cluster, each document write
> > > > will
> > > > be
> > > > written redundantly to every node, meaning CPU load will be
> > > > (mostly)
> > > > equal across all 3 nodes as you write that document.
> > > >
> > > > Hope this helps,
> > > > Joan
> > > >
> > > >
> > > > ----- Original Message -----
> > > > > From: "Joey Samonte" <csharpdeveloper@hotmail.com>
> > > > > To: dev@couchdb.apache.org
> > > > > Sent: Thursday, September 8, 2016 11:30:46 PM
> > > > > Subject: High CPU usage caused by clustering and/or
> > > > > replication?
> > > > >
> > > > > Good day,
> > > > >
> > > > > I have setup a cluster with two nodes, and I am replicating
> > > > > the
> > > > > first
> > > > > node to an existing database through a loadbalancer. But I am
> > > > > getting a 100% CPU usage after a few minutes on both nodes.
> > > > > Is
> > > > > this
> > > > > an issue with RC4?
> > > > >
> > > > > Thanks.
> > > > >
> > >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message