Mailing-List: contact dev-help@manifoldcf.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@manifoldcf.apache.org
Date: Sat, 7 Feb 2015 09:17:34 +0000 (UTC)
From: "Karl Wright (JIRA)" <jira@apache.org>
To: dev@manifoldcf.apache.org
Message-ID: <JIRA.12772398.1423064222000.290264.1423300654895@Atlassian.JIRA>
In-Reply-To: <JIRA.12772398.1423064222000@Atlassian.JIRA>
References: <JIRA.12772398.1423064222000@Atlassian.JIRA>
 <JIRA.12772398.1423064222139@arcas>
Subject: [jira] [Commented] (CONNECTORS-1156) Can't shut down agents service
 when automatic ANALYZE TABLE, or any long-running query, is happening
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit


    [ https://issues.apache.org/jira/browse/CONNECTORS-1156?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14310627#comment-14310627 ] 

Karl Wright commented on CONNECTORS-1156:
-----------------------------------------

I determined, with some effort, that Adrian's issue was due in part to the database being so badly in need of vacuuming that even basic query performance was poor.  Adrian has many many jobs as well, all on schedules, and so multiple jobs wind up being in the "starting up" state eventually under those conditions.  The problem in ManifoldCF here was that under certain error conditions, it did not *immediately* throw an INTERRUPTED ManifoldCFException up the stack, but instead attempted to process everything first before throwing the error.  That would cause failure shutting down, because *all* queries would fail.

This could also be the problem Aeham was seeing, because the same startup thread would be involved in initializing multiple jobs, which might also have a very extended initialization period.  However, Aeham's description of the context was not sufficiently detailed for me to be sure of this.

> Can't shut down agents service when automatic ANALYZE TABLE, or any long-running query, is happening
> ----------------------------------------------------------------------------------------------------
>
>                 Key: CONNECTORS-1156
>                 URL: https://issues.apache.org/jira/browse/CONNECTORS-1156
>             Project: ManifoldCF
>          Issue Type: Bug
>          Components: Framework agents process
>    Affects Versions: ManifoldCF 1.8, ManifoldCF 2.0
>            Reporter: Karl Wright
>            Assignee: Karl Wright
>             Fix For: ManifoldCF 1.9, ManifoldCF 2.1
>
>         Attachments: sqlstate.patch
>
>
> This was detected on MCF 1.8.
> The thread blocking shutdown had the following trace:
> {code}
> "Startup thread" daemon prio=10 tid=0x00007fe73012f000 nid=0x340b in Object.wait() [0x00007fe71f7f6000]
>    java.lang.Thread.State: WAITING (on object monitor)
>     at java.lang.Object.wait(Native Method)
>     at java.lang.Thread.join(Thread.java:1281)
>     - locked <0x00000000e851a0b8> (a org.apache.manifoldcf.core.database.Database$ExecuteQueryThread)
>     at java.lang.Thread.join(Thread.java:1355)
>     at org.apache.manifoldcf.core.database.Database$ExecuteQueryThread.finishUp(Database.java:694)
>     at org.apache.manifoldcf.core.database.Database.executeViaThread(Database.java:728)
>     at org.apache.manifoldcf.core.database.Database.executeUncachedQuery(Database.java:790)
>     at org.apache.manifoldcf.core.database.Database$QueryCacheExecutor.create(Database.java:1444)
>     at org.apache.manifoldcf.core.cachemanager.CacheManager.findObjectsAndExecute(CacheManager.java:146)
>     at org.apache.manifoldcf.core.database.Database.executeQuery(Database.java:191)
>     at org.apache.manifoldcf.core.database.DBInterfacePostgreSQL.performModification(DBInterfacePostgreSQL.java:656)
>     at org.apache.manifoldcf.core.database.DBInterfacePostgreSQL.analyzeTableInternal(DBInterfacePostgreSQL.java:1431)
>     at org.apache.manifoldcf.core.database.DBInterfacePostgreSQL.noteModificationsNoTransactions(DBInterfacePostgreSQL.java:1576)
>     at org.apache.manifoldcf.core.database.Database.playbackModifications(Database.java:429)
>     at org.apache.manifoldcf.core.database.Database.endTransaction(Database.java:414)
>     at org.apache.manifoldcf.core.database.DBInterfacePostgreSQL.endTransaction(DBInterfacePostgreSQL.java:1231)
>     at org.apache.manifoldcf.crawler.jobs.JobManager.resetStartupJob(JobManager.java:7575)
>     at org.apache.manifoldcf.crawler.system.StartupThread.run(StartupThread.java:238)
> {code}
> This was after many minutes of waiting for the shutdown to take place, so clearly it's looping in the sense that it keeps starting an ANALYZE, gets interrupted, and retries indefinitely.


--
This message was sent by Atlassian JIRA
(v6.3.4#6332)