Return-Path: 
 <user-return-10186-apmail-couchdb-user-archive=couchdb.apache.org@couchdb.apache.org>
Delivered-To: apmail-couchdb-user-archive@www.apache.org
Received: (qmail 89805 invoked from network); 19 Apr 2010 20:18:24 -0000
Received: from unknown (HELO mail.apache.org) (140.211.11.3)
  by 140.211.11.9 with SMTP; 19 Apr 2010 20:18:24 -0000
Received: (qmail 88574 invoked by uid 500); 19 Apr 2010 20:18:23 -0000
Delivered-To: apmail-couchdb-user-archive@couchdb.apache.org
Received: (qmail 88500 invoked by uid 500); 19 Apr 2010 20:18:23 -0000
Mailing-List: contact user-help@couchdb.apache.org; run by ezmlm
Precedence: bulk
List-Help: <mailto:user-help@couchdb.apache.org>
List-Unsubscribe: <mailto:user-unsubscribe@couchdb.apache.org>
List-Post: <mailto:user@couchdb.apache.org>
List-Id: <user.couchdb.apache.org>
Reply-To: user@couchdb.apache.org
Delivered-To: mailing list user@couchdb.apache.org
Received: (qmail 88490 invoked by uid 99); 19 Apr 2010 20:18:23 -0000
Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230)
    by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 19 Apr 2010 20:18:23 +0000
X-ASF-Spam-Status: No, hits=0.0 required=10.0
	tests=FREEMAIL_FROM,SPF_PASS,T_TO_NO_BRKTS_FREEMAIL
X-Spam-Check-By: apache.org
Received-SPF: pass (nike.apache.org: domain of adam.kocoloski@gmail.com
 designates 74.125.82.180 as permitted sender)
Received: from [74.125.82.180] (HELO mail-wy0-f180.google.com) (74.125.82.180)
    by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 19 Apr 2010 20:18:14 +0000
Received: by wyf22 with SMTP id 22so2811975wyf.11
        for <multiple recipients>; Mon, 19 Apr 2010 13:17:53 -0700 (PDT)
Received: by 10.216.173.74 with SMTP id u52mr1470092wel.204.1271708273634;
        Mon, 19 Apr 2010 13:17:53 -0700 (PDT)
Received: from [10.1.10.164] (c-66-31-20-188.hsd1.ma.comcast.net
 [66.31.20.188])
        by mx.google.com with ESMTPS id e82sm951882wej.4.2010.04.19.13.17.52
        (version=TLSv1/SSLv3 cipher=RC4-MD5);
        Mon, 19 Apr 2010 13:17:53 -0700 (PDT)
Subject: Re: SASL overhead (was Re: CouchDB and Hadoop_)
Mime-Version: 1.0 (Apple Message framework v1078)
Content-Type: text/plain; charset=us-ascii
From: Adam Kocoloski <kocolosk@apache.org>
In-Reply-To: <p2xec7a93a1004191303t4282a6c0j23da7128c3837a90@mail.gmail.com>
Date: Mon, 19 Apr 2010 16:17:50 -0400
Cc: dev@couchdb.apache.org
Reply-To: dev@couchdb.apache.org
Content-Transfer-Encoding: quoted-printable
Message-Id: <C667DAC5-EA9C-409E-BC87-F815A1BFD829@apache.org>
References: <p2xec7a93a1004191303t4282a6c0j23da7128c3837a90@mail.gmail.com>
To: user@couchdb.apache.org
X-Mailer: Apple Mail (2.1078)
X-Virus-Checked: Checked by ClamAV on apache.org

CouchDB definitely needs a bit of spring cleaning in the logging =
department.  If you haven't noticed, most error messages are written =
*twice* in the logs, once in a very nice format by SASL and once in a =
crude format by couch_log.  I believe this is because the couch_log =
event handler is installed in the same event manager as the SASL =
error_logger, and because the second handle_event clause in couch_log =
matches SASL-style error reports. So even if you turned off the SASL =
error_logger I think you'd still get these messages in the log.  I've =
been meaning to fix that ...

I'd prefer to keep the SASL-style logs, although I understand that we =
need to trim them down.  That can be accomplished on a =
process-by-process basis by identifying the processes with large =
internal states and truncating those states in a terminate() function =
just before exiting (perhaps keeping the full state if ?LOG_DEBUG is =
on).  Cheers,

Adam

On Apr 19, 2010, at 4:03 PM, Randall Leeds wrote:

> I've found that replication crashes pin the cpu and can even make =
couch
> unresponsive because the message queues and internal state of the
> replication processes is huge SASL logging serializes it all to the =
log
> file.
>=20
> I don't really have a good solution to this. Maybe there should be an =
option
> for turning off SASL in couch which would restrict the log messages to
> explicitly logged messages from couches ?LOG_<level> functions. It =
seems a
> reasonable default to me that SASL is only enabled when log level is =
DEBUG.
> If people feel good about this I'd happily make the patch.
>=20
> On Apr 19, 2010 10:39 AM, "Adam Kocoloski" <kocolosk@apache.org> =
wrote:
>=20
> Thanks Fredrik.  I think I have a pretty good handle on what's =
happening and
> have replied in detail in JIRA.  Best,
>=20
> Adam
>=20
>=20
> On Apr 19, 2010, at 10:22 AM, Fredrik Widlund wrote:
>=20
>>=20
>>=20
>> Hi,
>>=20
>> https://issues.apache.org/ji...