manifoldcf-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Karl Wright <daddy...@gmail.com>
Subject Re: max_pred_locks_per_transaction
Date Thu, 07 Feb 2013 14:34:21 GMT
FWIW, the way this diagnostic code works is that it simply dumps ALL
the handles that are currently considered to be in use.  Some of them
are leaked, and some are just being used.  So it's really the sum
total of all of them that will help us narrow down where leaks are
occurring.

I looked at the one location you provided; it's entirely unremarkable;
it's not even happening in a transaction.  If similar locations show
up again and again, it will mean that the problem must be in the part
of the code that allocates connections at the lowest level.  Before I
start picking that code apart line by line, though, I'd like all the
data.

Karl

On Thu, Feb 7, 2013 at 9:07 AM, Karl Wright <daddywri@gmail.com> wrote:
> Thanks, this is great.
>
> Could you include ALL of the 'possibly leaked' connections?  I'm
> looking for the ones that show up over and over again.  Please attach
> that part of the log to the ticket.
>
> Karl
>
> On Thu, Feb 7, 2013 at 9:04 AM, Erlend GarĂ¥sen <e.f.garasen@usit.uio.no> wrote:
>> On 06.02.13 15.24, Karl Wright wrote:
>>
>>> FWIW, schema creation has been working fine here, and all tests that
>>> exercise the schema have been passing fine as well.
>>
>>
>> My fault, but everything is successfully set up now.
>>
>> I think I have reproduced the issue. The crawl is still running, but I guess
>> it will stop very soon due to the following entries in my log. The last
>> fetched URL before the first of these log entries below was:
>> https://www.duo.uio.no/handle/123456789/31/discover?fq=author_filter%3Aaaberge%2C%5C+astrid%5C%7C%5C%7C%5C%7CAaberge%2C%5C+Astrid
>>
>> It seems that this URL is part of a search, so maybe we will get around this
>> problem by filtering out all search-related URLs. The problem is that it is
>> already filtered out, but by doing post filtering. I can try to adjust that
>> at next attempt.
>>
>>  INFO 2013-02-07 14:32:08,166 (Worker thread '13') - WEB: FETCH
>> URL|https://www.duo.uio.no/handle/123456789/31/discover?fq=author_filter%3Aaaberge%2C%5C+astrid%5C%7C%5C%7C%5C%7CAaberge%2C%5C+Astrid|1360243927960+204|200|24992|
>>  WARN 2013-02-07 14:32:08,228 (Worker thread '16') - Out of db connections,
>> list of outstanding ones follows.
>>  WARN 2013-02-07 14:32:08,229 (Worker thread '16') - Found a possibly leaked
>> db connection
>> java.lang.Exception: Possibly leaked db connection
>>         at
>> org.apache.manifoldcf.core.jdbcpool.ConnectionPool.getConnection(ConnectionPool.java:72)
>>
>>         at
>> org.apache.manifoldcf.core.database.ConnectionFactory.getConnectionWithRetries(ConnectionFactory.java:125)
>>         at
>> org.apache.manifoldcf.core.database.ConnectionFactory.getConnection(ConnectionFactory.java:96)
>>         at
>> org.apache.manifoldcf.core.database.Database.executeUncachedQuery(Database.java:758)
>>         at
>> org.apache.manifoldcf.core.database.Database$QueryCacheExecutor.create(Database.java:1430)
>>
>>         at
>> org.apache.manifoldcf.core.cachemanager.CacheManager.findObjectsAndExecute(CacheManager.java:144)
>>         at
>> org.apache.manifoldcf.core.database.Database.executeQuery(Database.java:186)
>>         at
>> org.apache.manifoldcf.core.database.DBInterfacePostgreSQL.performQuery(DBInterfacePostgreSQL.java:803)
>>         at
>> org.apache.manifoldcf.core.database.BaseTable.performQuery(BaseTable.java:221)
>>         at
>> org.apache.manifoldcf.crawler.jobs.HopCount.processFind(HopCount.java:674)
>>         at
>> org.apache.manifoldcf.crawler.jobs.HopCount.findHopCounts(HopCount.java:649)
>>         at
>> org.apache.manifoldcf.crawler.jobs.JobManager.findHopCounts(JobManager.java:3777)
>>         at
>> org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThread.java:224)
>>  WARN 2013-02-07 14:32:08,233 (Worker thread '16') - Found a possibly leaked
>> db connection
>>
>>
>>
>>
>> --
>> Erlend GarĂ¥sen
>> Center for Information Technology Services
>> University of Oslo
>> P.O. Box 1086 Blindern, N-0317 OSLO, Norway
>> Ph: (+47) 22840193, Fax: (+47) 22852970, Mobile: (+47) 91380968, VIP: 31050

Mime
View raw message