manifoldcf-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Karl Wright (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (CONNECTORS-1156) Can't shut down agents service when automatic ANALYZE TABLE, or any long-running query, is happening
Date Sat, 07 Feb 2015 09:17:34 GMT

    [ https://issues.apache.org/jira/browse/CONNECTORS-1156?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14310627#comment-14310627
] 

Karl Wright commented on CONNECTORS-1156:
-----------------------------------------

I determined, with some effort, that Adrian's issue was due in part to the database being
so badly in need of vacuuming that even basic query performance was poor.  Adrian has many
many jobs as well, all on schedules, and so multiple jobs wind up being in the "starting up"
state eventually under those conditions.  The problem in ManifoldCF here was that under certain
error conditions, it did not *immediately* throw an INTERRUPTED ManifoldCFException up the
stack, but instead attempted to process everything first before throwing the error.  That
would cause failure shutting down, because *all* queries would fail.

This could also be the problem Aeham was seeing, because the same startup thread would be
involved in initializing multiple jobs, which might also have a very extended initialization
period.  However, Aeham's description of the context was not sufficiently detailed for me
to be sure of this.

> Can't shut down agents service when automatic ANALYZE TABLE, or any long-running query,
is happening
> ----------------------------------------------------------------------------------------------------
>
>                 Key: CONNECTORS-1156
>                 URL: https://issues.apache.org/jira/browse/CONNECTORS-1156
>             Project: ManifoldCF
>          Issue Type: Bug
>          Components: Framework agents process
>    Affects Versions: ManifoldCF 1.8, ManifoldCF 2.0
>            Reporter: Karl Wright
>            Assignee: Karl Wright
>             Fix For: ManifoldCF 1.9, ManifoldCF 2.1
>
>         Attachments: sqlstate.patch
>
>
> This was detected on MCF 1.8.
> The thread blocking shutdown had the following trace:
> {code}
> "Startup thread" daemon prio=10 tid=0x00007fe73012f000 nid=0x340b in Object.wait() [0x00007fe71f7f6000]
>    java.lang.Thread.State: WAITING (on object monitor)
>     at java.lang.Object.wait(Native Method)
>     at java.lang.Thread.join(Thread.java:1281)
>     - locked <0x00000000e851a0b8> (a org.apache.manifoldcf.core.database.Database$ExecuteQueryThread)
>     at java.lang.Thread.join(Thread.java:1355)
>     at org.apache.manifoldcf.core.database.Database$ExecuteQueryThread.finishUp(Database.java:694)
>     at org.apache.manifoldcf.core.database.Database.executeViaThread(Database.java:728)
>     at org.apache.manifoldcf.core.database.Database.executeUncachedQuery(Database.java:790)
>     at org.apache.manifoldcf.core.database.Database$QueryCacheExecutor.create(Database.java:1444)
>     at org.apache.manifoldcf.core.cachemanager.CacheManager.findObjectsAndExecute(CacheManager.java:146)
>     at org.apache.manifoldcf.core.database.Database.executeQuery(Database.java:191)
>     at org.apache.manifoldcf.core.database.DBInterfacePostgreSQL.performModification(DBInterfacePostgreSQL.java:656)
>     at org.apache.manifoldcf.core.database.DBInterfacePostgreSQL.analyzeTableInternal(DBInterfacePostgreSQL.java:1431)
>     at org.apache.manifoldcf.core.database.DBInterfacePostgreSQL.noteModificationsNoTransactions(DBInterfacePostgreSQL.java:1576)
>     at org.apache.manifoldcf.core.database.Database.playbackModifications(Database.java:429)
>     at org.apache.manifoldcf.core.database.Database.endTransaction(Database.java:414)
>     at org.apache.manifoldcf.core.database.DBInterfacePostgreSQL.endTransaction(DBInterfacePostgreSQL.java:1231)
>     at org.apache.manifoldcf.crawler.jobs.JobManager.resetStartupJob(JobManager.java:7575)
>     at org.apache.manifoldcf.crawler.system.StartupThread.run(StartupThread.java:238)
> {code}
> This was after many minutes of waiting for the shutdown to take place, so clearly it's
looping in the sense that it keeps starting an ANALYZE, gets interrupted, and retries indefinitely.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message