Return-Path: X-Original-To: apmail-cayenne-user-archive@www.apache.org Delivered-To: apmail-cayenne-user-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id CC2A318A42 for ; Mon, 7 Mar 2016 20:29:12 +0000 (UTC) Received: (qmail 44981 invoked by uid 500); 7 Mar 2016 20:29:12 -0000 Delivered-To: apmail-cayenne-user-archive@cayenne.apache.org Received: (qmail 44951 invoked by uid 500); 7 Mar 2016 20:29:12 -0000 Mailing-List: contact user-help@cayenne.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@cayenne.apache.org Delivered-To: mailing list user@cayenne.apache.org Received: (qmail 44936 invoked by uid 99); 7 Mar 2016 20:29:12 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd2-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 07 Mar 2016 20:29:12 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd2-us-west.apache.org (ASF Mail Server at spamd2-us-west.apache.org) with ESMTP id C95DF1A04CF for ; Mon, 7 Mar 2016 20:29:11 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd2-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: 0.999 X-Spam-Level: X-Spam-Status: No, score=0.999 tagged_above=-999 required=6.31 tests=[KAM_LAZY_DOMAIN_SECURITY=1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-0.001] autolearn=disabled Received: from mx2-lw-eu.apache.org ([10.40.0.8]) by localhost (spamd2-us-west.apache.org [10.40.0.9]) (amavisd-new, port 10024) with ESMTP id fmSAGpC7z5o6 for ; Mon, 7 Mar 2016 20:29:09 +0000 (UTC) Received: from mail-03.1984.is (mail-03.1984.is [93.95.224.70]) by mx2-lw-eu.apache.org (ASF Mail Server at mx2-lw-eu.apache.org) with ESMTPS id 40EAF5F54F for ; Mon, 7 Mar 2016 20:29:09 +0000 (UTC) Received: from localhost by mail-03.1984.is with esmtpsa (TLSv1:DHE-RSA-AES256-SHA:256) (Exim 4.84) (envelope-from ) id 1ad1m8-0005XR-R7 for user@cayenne.apache.org; Mon, 07 Mar 2016 20:29:08 +0000 Content-Type: text/plain; charset=utf-8 Mime-Version: 1.0 (Mac OS X Mail 9.2 \(3112\)) Subject: Re: Reconnecting automatically after DB downtime From: Hugi Thordarson In-Reply-To: <135D44F4-0F99-4286-BF84-ED3404B1BF01@karlmenn.is> Date: Mon, 7 Mar 2016 20:29:08 +0000 Content-Transfer-Encoding: quoted-printable Message-Id: References: <15E01460-2B2C-4E0A-BE44-86D8020060F7@karlmenn.is> <586DD623-7C0F-4AF9-80C5-53DDB80A6ADA@karlmenn.is> <55049F21-3527-4B88-90BC-202D1451EDB7@objectstyle.org> <1CE7D336-E62F-443E-B6A1-4FC9863FC94F@karlmenn.is> <135D44F4-0F99-4286-BF84-ED3404B1BF01@karlmenn.is> To: user@cayenne.apache.org X-Mailer: Apple Mail (2.3112) Is it possible it=E2=80=99s a configuration issue? I=E2=80=99m setting = minConnections to 1 and maxConnections to 10. Should I rather leave = these at default values or choose different values? Cheers, - hugi // Hugi Thordarson // http://www.loftfar.is/ // s. 895-6688 > On 7. mar. 2016, at 20:01, Hugi Thordarson wrote: >=20 > Thanks for the reply Andrus! >=20 > Sounds like it might be the case. But actually, this now happens = without the DB going down (and I don=E2=80=99t think it=E2=80=99s a = communication problem with the DB either, since the DB runs on the same = server as the apps). >=20 > I recently added a PostCommitListener that does quite a lot of DB work = and I believe the issue started around the same time. At first I thought = it might be a deadlocking sitation, but a jstack thread dump from a dead = process does not point to that. >=20 > Is there any way for me to check the number and status of currently = open connections? >=20 > Cheers, > - hugi >=20 >=20 >=20 >> On 7. mar. 2016, at 18:54, Andrus Adamchik = wrote: >>=20 >> Hmm... It shows there are zero connections in the pool. I have a = theory about that. Looks like when you uncheck your first connection = after the DB restart, pool goes through all connections to find a valid = one, and removing bad connections in the process. As there are no longer = valid connections there, the pool is drained completely. Then the error = happens; and then at a later time, pool manager thread kicks in and = fills the pool with new connections.=20 >>=20 >> I need to test it.. And perhaps move the validation to background.=20 >>=20 >> (and perhaps add Tomcat connection pool to a list of DataSource = options ... that's the one I am using in Bootique: = https://github.com/nhl/bootique-jdbc and elsewhere in production). >>=20 >> Andrus >>=20 >>> On Mar 7, 2016, at 2:46 AM, Hugi Thordarson = wrote: >>>=20 >>> Hi all, >>> This is still happening, even after I added a validationQuery, our = app is dying quite frequently :(. I=E2=80=99m not quite sure how to = debug this, is there any way for me to catch where connections are being = opened in the code and at what location they=E2=80=99re hanging? >>>=20 >>> = https://www.dropbox.com/s/8jkmh6513s6wwkn/Screenshot%202016-03-07%2010.21.= 53.png?dl=3D0 >>>=20 >>> Cheer, >>> - hugi >>>=20 >>> // Hugi Thordarson >>> // http://www.loftfar.is/ >>> // s. 895-6688 >>>=20 >>>=20 >>>=20 >>>> On 29. feb. 2016, at 11:25, Andrus Adamchik = wrote: >>>>=20 >>>>=20 >>>>> On Feb 29, 2016, at 2:20 PM, Hugi Thordarson = wrote: >>>>>=20 >>>>> What does validationQuery do? >>>>=20 >>>> Periodically executes for each pooled connection, and kills = connections that throw during validation. So it ensures that all pooled = connections are in a good state. >>>>=20 >>>> Andrus >>>>=20 >>>=20 >>=20 >=20