Mailing-List: contact user-help@cayenne.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@cayenne.apache.org
MIME-Version: 1.0
In-Reply-To: <272C0255-17C0-4353-B95C-941B84A9F43C@objectstyle.org>
References: 
 <CAM1yOjaMT5vT8B-_FoYLP0bqvrdq7zMhGUn_swO8+CxTpdBJgg@mail.gmail.com>
 <1EF83356-E7BD-4438-88E2-30D222466E75@objectstyle.org>
 <CAM1yOjYe67h3cMBoV6YoH0oP6xhSp2t=7c6BSpS8saMkGx_3cA@mail.gmail.com>
 <CAM1yOja9Nac9cFOvXEHxyJyjASN_q0AgDok46Q8uvQAyWkdaUQ@mail.gmail.com>
 <272C0255-17C0-4353-B95C-941B84A9F43C@objectstyle.org>
From: Mike Kienenberger <mkienenb@gmail.com>
Date: Fri, 12 Jun 2015 11:09:37 -0400
Message-ID: 
 <CAM1yOjYHNTgHX_fKmAnOQC4QxhGxJTjFtqqfT8ZXwjD3EqNROQ@mail.gmail.com>
Subject: Re: java.sql.SQLException: Can't obtain connection. Request timed
 out.
To: user@cayenne.apache.org
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: quoted-printable

Thanks for the pointer to jstack.  Looking through the output now, but
the problem doesn't persist.   I've only had it happen twice in 3 or 4
days.

I guess I need to dump a full all-thread stacktrace like jstack when
the problem occurs to see what is going on.

I'm also looking through the poolmanager source to see if there's any
additional logging I could turn on, but I'm not seeing anything
cayenne-related executing at the moment.

And, interestingly enough, I only have 8 "PoolManagerCleanup" threads,
and not the 9 that I expected.  Maybe they are create-on-demand and
one of my runtimes hasn't needed it yet?


On Fri, Jun 12, 2015 at 10:56 AM, Andrus Adamchik
<andrus@objectstyle.org> wrote:
> So there's a leak somewhere. Do you see any hanging threads with jstack, =
or maybe some unclosed ResultIterators?
>
> Andrus
>
>> On Jun 12, 2015, at 5:51 PM, Mike Kienenberger <mkienenb@gmail.com> wrot=
e:
>>
>> ....And upping my connection pool size from 1 to 3 didn't help as I
>> got the error again today, but this time with all 3 in use.
>>
>> Caused by: java.sql.SQLException: Can't obtain connection. Request
>> timed out. Total used connections: 3
>>
>>
>>
>> On Thu, Jun 11, 2015 at 11:41 AM, Mike Kienenberger <mkienenb@gmail.com>=
 wrote:
>>> So what I had when I originally deployed would have been three pools
>>> of one connection each?   That's fine.  And that explains why it
>>> worked without problems.
>>>
>>> So when I increased to 3 connections, I actually have 9 now, three in
>>> each of the three pools which I definitely do not want.
>>>
>>> So apparently I only hit this problem in a single runtime's connection
>>> pool, and not in all of them.   That should make it easier to track
>>> down what caused it.   Thanks.
>>>
>>>
>>>
>>> On Thu, Jun 11, 2015 at 11:32 AM, Andrus Adamchik
>>> <andrus@objectstyle.org> wrote:
>>>>> My guess is that all three runtimes share the same pool
>>>>
>>>>
>>>> Actually no. If you have 3 runtimes, and let Cayenne instantiate its p=
ool (as the stack trace implies) instead of using say container JNDI DataSo=
urce, you should have 3 instances of the pool. You can try confirming the n=
umber of PoolManager instances with VisualVM or jmap.
>>>>
>>>> Andrus
>>>>
>>>>> On Jun 11, 2015, at 6:18 PM, Mike Kienenberger <mkienenb@gmail.com> w=
rote:
>>>>>
>>>>> So I finally put my old Cayenne 1.x-upgraded to 3.x app into producti=
on.
>>>>>
>>>>> A couple of hours later, I got this error [1].   Note that my app
>>>>> synchronizes requests so only one thread should ever be running
>>>>> Cayenne requests at a time.
>>>>>
>>>>> My app has run for years with only a single connection.   My 3.x app
>>>>> ran in development for a year before it was deployed to production.
>>>>>
>>>>> Is this a fluke?   I did raise my connection pool to 3 the next day,
>>>>> but I don't want to arbitrarily assign a higher number and think I've
>>>>> magically solved the problem.
>>>>>
>>>>> Does having three active server runtimes affect this?   Before, my
>>>>> query qualifiers were all handled inside the main runtime, but now I
>>>>> have three separate runtimes configured.   My guess is that all three
>>>>> runtimes share the same pool, so this is probably not the cause.   An=
d
>>>>> since the problem happened a couple hours later and the three runtime=
s
>>>>> were being used well before that without an error, I'd say experience
>>>>> bears this out.
>>>>>
>>>>>
>>>>> [1]
>>>>>
>>>>> Caused by: java.sql.SQLException: Can't obtain connection. Request
>>>>> timed out. Total used connections: 1
>>>>>       at org.apache.cayenne.conn.PoolManager.uncheckPooledConnection(=
PoolManager.java:452)
>>>>>       at org.apache.cayenne.conn.PoolManager.getConnection(PoolManage=
r.java:382)
>>>>>       at org.apache.cayenne.conn.PoolManager.getConnection(PoolManage=
r.java:371)
>>>>>       at org.apache.cayenne.access.DataNode$TransactionDataSource.get=
Connection(DataNode.java:351)
>>>>>       at org.apache.cayenne.access.DataNode.performQueries(DataNode.j=
ava:259)
>>>>>
>>>>
>>
>