From user-return-20401-apmail-couchdb-user-archive=couchdb.apache.org@couchdb.apache.org Wed Apr 11 17:56:30 2012 Return-Path: X-Original-To: apmail-couchdb-user-archive@www.apache.org Delivered-To: apmail-couchdb-user-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id E921A98B7 for ; Wed, 11 Apr 2012 17:56:30 +0000 (UTC) Received: (qmail 53378 invoked by uid 500); 11 Apr 2012 17:56:29 -0000 Delivered-To: apmail-couchdb-user-archive@couchdb.apache.org Received: (qmail 53247 invoked by uid 500); 11 Apr 2012 17:56:29 -0000 Mailing-List: contact user-help@couchdb.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@couchdb.apache.org Delivered-To: mailing list user@couchdb.apache.org Received: (qmail 53239 invoked by uid 99); 11 Apr 2012 17:56:29 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 11 Apr 2012 17:56:29 +0000 X-ASF-Spam-Status: No, hits=-0.5 required=5.0 tests=FREEMAIL_ENVFROM_END_DIGIT,RCVD_IN_DNSWL_LOW,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (athena.apache.org: domain of elf2001@gmail.com designates 209.85.213.180 as permitted sender) Received: from [209.85.213.180] (HELO mail-yx0-f180.google.com) (209.85.213.180) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 11 Apr 2012 17:56:23 +0000 Received: by yenl4 with SMTP id l4so801261yen.11 for ; Wed, 11 Apr 2012 10:56:02 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type; bh=OrmS9rq0xX2xPLKJcIOxnQ+fcJffOk31+cESNQBsJlA=; b=ap35Rgi02DgCg03E4IHwBexo3bNRJrVGLQrmZBk9Rc0h9OwdFzI0EabWfxrMqGC/IG kCfVo4dSkwmPxfx5ZH+B0ncMACfyKOc6YTr4swt+8PMzl1TqUD1/B4wD3Jl4ZAPFAEqE I97MVHNQ2r4gWBMaHNbSxYQlk1LGXZQ91DsqeUbRtymrM/mr5hiH9FIjPkpUVNhsyt8O 3XlNP1dcHng9xjEHwLVdqG3s9TXyNSl3eQiPp2oj5ScvT1HS+/ysaX1n/BRzx68mUs0T R70BAAgpQL9yuXdcfvEmoYmBzQgvyyZI0aiRwZWpEACjy+Hl6TYFj1gbitCVtyUYS7bt 8b8Q== MIME-Version: 1.0 Received: by 10.50.180.135 with SMTP id do7mr6630221igc.56.1334166962172; Wed, 11 Apr 2012 10:56:02 -0700 (PDT) Received: by 10.42.133.6 with HTTP; Wed, 11 Apr 2012 10:56:02 -0700 (PDT) In-Reply-To: References: Date: Wed, 11 Apr 2012 20:56:02 +0300 Message-ID: Subject: Re: replication with 1.2.0 cause excess GET requests for databases From: Alexey Loshkarev To: user@couchdb.apache.org Content-Type: text/plain; charset=ISO-8859-1 X-Virus-Checked: Checked by ClamAV on apache.org I've realized now, CouchDB ate all my cpu. I see 4 beam.smp in htop processes eating 100% CPU each (i have 4 cores). But I have not run any cpu-consuming tasks like view indexing. 2012/4/11 Alexey Loshkarev : > I've updated to 1.2.0 on my production set (2 master-master nodes with > several databases). > > I have 2 pull replication of each database (6 databases at each node) > between masters. > > After upgrading, i ran replication (via _replication DB). > > 4 of databases (3 small and 1 about 300k documents) have replicated in > few minutes and two other replicates very slow. Speed is about 100 > changes/second (it was about 10-20k changes/second). > I've tried to do custom replication (without _replication db), > continuous or not - the same - very-very slow replication process. > Also, I seeing excess GET-requests on both nodes: > > [Wed, 11 Apr 2012 17:34:35 GMT] [info] [<0.12727.7>] 10.0.0.42 - - GET > /exhaust_orders/ 200 > [Wed, 11 Apr 2012 17:34:35 GMT] [info] [<0.31528.6>] 10.0.0.42 - - GET > /exhaust_events/ 200 > [Wed, 11 Apr 2012 17:34:35 GMT] [info] [<0.12727.7>] 10.0.0.42 - - GET > /exhaust_orders/ 200 > [Wed, 11 Apr 2012 17:34:35 GMT] [info] [<0.31528.6>] 10.0.0.42 - - GET > /exhaust_events/ 200 > [Wed, 11 Apr 2012 17:34:35 GMT] [info] [<0.12727.7>] 10.0.0.42 - - GET > /exhaust_orders/ 200 > [Wed, 11 Apr 2012 17:34:35 GMT] [info] [<0.31528.6>] 10.0.0.42 - - GET > /exhaust_events/ 200 > [Wed, 11 Apr 2012 17:34:35 GMT] [info] [<0.12727.7>] 10.0.0.42 - - GET > /exhaust_orders/ 200 > [Wed, 11 Apr 2012 17:34:35 GMT] [info] [<0.31528.6>] 10.0.0.42 - - GET > /exhaust_events/ 200 > [Wed, 11 Apr 2012 17:34:35 GMT] [info] [<0.12727.7>] 10.0.0.42 - - GET > /exhaust_orders/ 200 > [Wed, 11 Apr 2012 17:34:35 GMT] [info] [<0.31528.6>] 10.0.0.42 - - GET > /exhaust_events/ 200 > [Wed, 11 Apr 2012 17:34:35 GMT] [info] [<0.31528.6>] 10.0.0.42 - - GET > /exhaust_events/ 200 > [Wed, 11 Apr 2012 17:34:35 GMT] [info] [<0.12727.7>] 10.0.0.42 - - GET > /exhaust_orders/ 200 > [Wed, 11 Apr 2012 17:34:35 GMT] [info] [<0.31528.6>] 10.0.0.42 - - GET > /exhaust_events/ 200 > [Wed, 11 Apr 2012 17:34:35 GMT] [info] [<0.12727.7>] 10.0.0.42 - - GET > /exhaust_orders/ 200 > [Wed, 11 Apr 2012 17:34:35 GMT] [info] [<0.14945.7>] 127.0.0.1 - - GET / 200 > [Wed, 11 Apr 2012 17:34:35 GMT] [info] [<0.31528.6>] 10.0.0.42 - - GET > /exhaust_events/ 200 > [Wed, 11 Apr 2012 17:34:35 GMT] [info] [<0.12727.7>] 10.0.0.42 - - GET > /exhaust_orders/ 200 > [Wed, 11 Apr 2012 17:34:35 GMT] [info] [<0.31528.6>] 10.0.0.42 - - GET > /exhaust_events/ 200 > [Wed, 11 Apr 2012 17:34:35 GMT] [info] [<0.12727.7>] 10.0.0.42 - - GET > /exhaust_orders/ 200 > [Wed, 11 Apr 2012 17:34:35 GMT] [info] [<0.31528.6>] 10.0.0.42 - - GET > /exhaust_events/ 200 > [Wed, 11 Apr 2012 17:34:35 GMT] [info] [<0.12727.7>] 10.0.0.42 - - GET > /exhaust_orders/ 200 > [Wed, 11 Apr 2012 17:34:35 GMT] [info] [<0.399.0>] recording a > checkpoint for `http://node3:5984/exhaust_orders/` -> `exhaust_orders` > at source update_seq 6482987 > [Wed, 11 Apr 2012 17:34:35 GMT] [info] [<0.31528.6>] 10.0.0.42 - - GET > /exhaust_events/ 200 > [Wed, 11 Apr 2012 17:34:35 GMT] [info] [<0.12727.7>] 10.0.0.42 - - GET > /exhaust_orders/ 200 > [Wed, 11 Apr 2012 17:34:35 GMT] [info] [<0.12727.7>] 10.0.0.42 - - GET > /exhaust_orders/ 200 > [Wed, 11 Apr 2012 17:34:35 GMT] [info] [<0.31528.6>] 10.0.0.42 - - GET > /exhaust_events/ 200 > [Wed, 11 Apr 2012 17:34:36 GMT] [info] [<0.12727.7>] 10.0.0.42 - - GET > /exhaust_orders/ 200 > [Wed, 11 Apr 2012 17:34:36 GMT] [info] [<0.31528.6>] 10.0.0.42 - - GET > /exhaust_events/ 200 > [Wed, 11 Apr 2012 17:34:36 GMT] [info] [<0.12727.7>] 10.0.0.42 - - GET > /exhaust_orders/ 200 > [Wed, 11 Apr 2012 17:34:36 GMT] [info] [<0.31528.6>] 10.0.0.42 - - GET > /exhaust_events/ 200 > [Wed, 11 Apr 2012 17:34:36 GMT] [info] [<0.12727.7>] 10.0.0.42 - - GET > /exhaust_orders/ 200 > [Wed, 11 Apr 2012 17:34:36 GMT] [info] [<0.31528.6>] 10.0.0.42 - - GET > /exhaust_events/ 200 > [Wed, 11 Apr 2012 17:34:36 GMT] [info] [<0.12727.7>] 10.0.0.42 - - GET > /exhaust_orders/ 200 > [Wed, 11 Apr 2012 17:34:36 GMT] [info] [<0.31528.6>] 10.0.0.42 - - GET > /exhaust_events/ 200 > [Wed, 11 Apr 2012 17:34:36 GMT] [info] [<0.31528.6>] 10.0.0.42 - - GET > /exhaust_events/ 200 > [Wed, 11 Apr 2012 17:34:36 GMT] [info] [<0.12727.7>] 10.0.0.42 - - GET > /exhaust_orders/ 200 > [Wed, 11 Apr 2012 17:34:36 GMT] [info] [<0.31528.6>] 10.0.0.42 - - GET > /exhaust_events/ 200 > [Wed, 11 Apr 2012 17:34:36 GMT] [info] [<0.12727.7>] 10.0.0.42 - - GET > /exhaust_orders/ 200 > [Wed, 11 Apr 2012 17:34:36 GMT] [info] [<0.32074.6>] recording a > checkpoint for `http://node2:5984/exhaust_events/` -> `exhaust_events` > at source update_seq 4557085 > [Wed, 11 Apr 2012 17:34:36 GMT] [info] [<0.31528.6>] 10.0.0.42 - - GET > /exhaust_events/ 200 > [Wed, 11 Apr 2012 17:34:36 GMT] [info] [<0.137.0>] recording a > checkpoint for `http://node3:5984/exhaust_queues/` -> `exhaust_queues` > at source update_seq 3371674 > [Wed, 11 Apr 2012 17:34:36 GMT] [info] [<0.31528.6>] 10.0.0.42 - - GET > /exhaust_events/ 200 > [Wed, 11 Apr 2012 17:34:36 GMT] [info] [<0.12727.7>] 10.0.0.42 - - > POST /exhaust_orders/_ensure_full_commit 201 > ... > > and so on > > so, it's about 10-20 rps on both nodes. > IP 10.0.0.42 - is node2. > Node1 has the same traffic in couchdb.log > > exhaust_orders and exhaust_events are last two databases. > > What happens with couchdb? Why it requests that databases so fast? > > > -- > ---------------- > Best regards > Alexey Loshkarev > mailto:elf2001@gmail.com -- ---------------- Best regards Alexey Loshkarev mailto:elf2001@gmail.com