From dev-return-19349-archive-asf-public=cust-asf.ponee.io@manifoldcf.apache.org Tue Mar 19 04:19:14 2019 Return-Path: X-Original-To: archive-asf-public@cust-asf.ponee.io Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by mx-eu-01.ponee.io (Postfix) with SMTP id 9FC20180651 for ; Tue, 19 Mar 2019 05:19:13 +0100 (CET) Received: (qmail 21603 invoked by uid 500); 19 Mar 2019 04:19:12 -0000 Mailing-List: contact dev-help@manifoldcf.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@manifoldcf.apache.org Delivered-To: mailing list dev@manifoldcf.apache.org Received: (qmail 21592 invoked by uid 99); 19 Mar 2019 04:19:12 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd4-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 19 Mar 2019 04:19:12 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd4-us-west.apache.org (ASF Mail Server at spamd4-us-west.apache.org) with ESMTP id 2C01AC078C for ; Tue, 19 Mar 2019 04:19:12 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd4-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: -109.5 X-Spam-Level: X-Spam-Status: No, score=-109.5 tagged_above=-999 required=6.31 tests=[ENV_AND_HDR_SPF_MATCH=-0.5, KAM_ASCII_DIVIDERS=0.8, RCVD_IN_DNSWL_MED=-2.3, SPF_PASS=-0.001, URIBL_BLOCKED=0.001, USER_IN_DEF_SPF_WL=-7.5, USER_IN_WHITELIST=-100] autolearn=disabled Received: from mx1-lw-eu.apache.org ([10.40.0.8]) by localhost (spamd4-us-west.apache.org [10.40.0.11]) (amavisd-new, port 10024) with ESMTP id s8lp1yWLLqkL for ; Tue, 19 Mar 2019 04:19:10 +0000 (UTC) Received: from mailrelay1-us-west.apache.org (mailrelay1-us-west.apache.org [209.188.14.139]) by mx1-lw-eu.apache.org (ASF Mail Server at mx1-lw-eu.apache.org) with ESMTP id 41F085F6C6 for ; Tue, 19 Mar 2019 04:12:01 +0000 (UTC) Received: from jira-lw-us.apache.org (unknown [207.244.88.139]) by mailrelay1-us-west.apache.org (ASF Mail Server at mailrelay1-us-west.apache.org) with ESMTP id 78DA6E00A7 for ; Tue, 19 Mar 2019 04:12:00 +0000 (UTC) Received: from jira-lw-us.apache.org (localhost [127.0.0.1]) by jira-lw-us.apache.org (ASF Mail Server at jira-lw-us.apache.org) with ESMTP id 3648024421 for ; Tue, 19 Mar 2019 04:12:00 +0000 (UTC) Date: Tue, 19 Mar 2019 04:12:00 +0000 (UTC) From: "Subasini Rath (JIRA)" To: dev@manifoldcf.apache.org Message-ID: In-Reply-To: References: Subject: [jira] [Commented] (CONNECTORS-880) Under the right conditions, job aborts do not update "last checked" time MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 [ https://issues.apache.org/jira/browse/CONNECTORS-880?page=3Dcom.atlas= sian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=3D= 16795657#comment-16795657 ]=20 Subasini Rath commented on CONNECTORS-880: ------------------------------------------ Hi Karl, =C2=A0 =C2=A0I am also facing the above mentioned issue. (Similar to Connec= tor-880) I am using manifold2.12 binary version. I am using Solr output connector an= d Web repository connection. Manifold is using all default configuration. When I am running the jobs manually, it runs fine. Same jobs have been sche= duled to run everyday. I am getting below exceptions and the job gets hanged/ going to waiting sta= ge. Could you please help me in resolving the same. I am getting the below error - Scenario-1 WARN 2019-03-08T23:58:20,338 (qtp550147359-413) - Found a long-running quer= y (2706114 ms):=C2=A0[SELECT t0.id,t0.description,t0.status,t0.starttime,t0= .endtime,t0.errortext FROM jobs t0 ORDER BY description ASC] WARN 2019-03-08T23:58:20,337 (Document delete stuffer thread) - Found a lon= g-running query (2737370 ms):=C2=A0[SELECT id FROM jobs WHERE status=3D? LI= MIT 1] WARN 2019-03-08T23:58:20,339 (Job reset thread) - Found a long-running quer= y (2770133 ms):=C2=A0[SELECT id FROM jobs WHERE status IN (?,?)] WARN 2019-03-08T23:58:20,386 (Document delete stuffer thread) - Parameter 0= : 'e' WARN 2019-03-08T23:58:20,337 (Set priority thread) - Found a long-running q= uery (2732379 ms):=C2=A0[SELECT id,dochash,docid,jobid FROM jobqueue WHERE = needpriority=3D? LIMIT 1000] WARN 2019-03-08T23:58:20,386 (Set priority thread) - Parameter 0: 'T' WARN 2019-03-08T23:58:20,386 (Job reset thread) - Parameter 0: 'I' WARN 2019-03-08T23:58:20,386 (Job reset thread) - Parameter 1: 'i' WARN 2019-03-08T23:58:20,372 (Seeding thread) - Parameter 2: '1552047176062= ' WARN 2019-03-08T23:58:20,474 (Document cleanup stuffer thread) - Found a lo= ng-running query (2737524 ms):=C2=A0[SELECT id FROM jobs WHERE status=3D? L= IMIT 1] WARN 2019-03-08T23:58:20,474 (Document cleanup stuffer thread) - Parameter = 0: 'S' WARN 2019-03-08T23:58:20,474 (Finisher thread) - Found a long-running query= (2752034 ms):=C2=A0[SELECT id FROM jobs WHERE status IN (?,?,?) FOR UPDATE= ] WARN 2019-03-08T23:58:20,474 (Finisher thread) - Parameter 0: 'A' WARN 2019-03-08T23:58:20,475 (Finisher thread) - Parameter 1: 'W' WARN 2019-03-08T23:58:20,475 (Finisher thread) - Parameter 2: 'R' WARN 2019-03-08T23:58:20,475 (Delete startup thread) - Found a long-running= query (2752036 ms):=C2=A0[SELECT id FROM jobs WHERE status=3D? FOR UPDATE] WARN 2019-03-08T23:58:20,475 (Delete startup thread) - Parameter 0: 'E' WARN 2019-03-08T23:58:20,483 (qtp550147359-4339) - Found a long-running que= ry (2496641 ms):=C2=A0[SELECT t0.id,t0.description,t0.status,t0.starttime,t= 0.endtime,t0.errortext FROM jobs t0 ORDER BY description ASC] WARN 2019-03-08T23:58:20,492 (Finisher thread) - Plan: isDistinctSelect=3D[= false] WARN 2019-03-08T23:58:20,492 (Finisher thread) - Plan: isGrouped=3D[false] WARN 2019-03-08T23:58:20,492 (Finisher thread) - Plan: isAggregated=3D[fals= e] WARN 2019-03-08T23:58:20,492 (Finisher thread) - Plan: columns=3D[ COLUMN: = PUBLIC.JOBS.ID not nullable WARN 2019-03-08T23:58:20,492 (qtp550147359-4346) - Found a long-running que= ry (2435908 ms):=C2=A0[SELECT t0.id,t0.description,t0.status,t0.starttime,t= 0.endtime,t0.errortext FROM jobs t0 ORDER BY description ASC] WARN 2019-03-08T23:58:20,492 (Finisher thread) - Plan:=C2=A0 WARN 2019-03-08T23:58:20,492 (Finisher thread) - Plan: ] WARN 2019-03-08T23:58:20,499 (Finisher thread) - Plan: [range variable 1 WARN 2019-03-08T23:58:20,499 (Finisher thread) - Plan: join type=3DINNER WARN 2019-03-08T23:58:20,499 (Finisher thread) - Plan: table=3DSYSTEM_SUBQU= ERY WARN 2019-03-08T23:58:20,499 (Finisher thread) - Plan: cardinality=3D0 WARN 2019-03-08T23:58:20,499 (Finisher thread) - Plan: access=3DFULL SCAN WARN 2019-03-08T23:58:20,500 (Finisher thread) - Plan: join condition =3D [= index=3DSYS_IDX_13329 WARN 2019-03-08T23:58:20,500 (Finisher thread) - Plan: ] WARN 2019-03-08T23:58:20,500 (Finisher thread) - Plan: ][range variable 2 WARN 2019-03-08T23:58:20,500 (Finisher thread) - Plan: join type=3DINNER WARN 2019-03-08T23:58:20,500 (Finisher thread) - Plan: table=3DJOBS WARN 2019-03-08T23:58:20,500 (Finisher thread) - Plan: cardinality=3D3 WARN 2019-03-08T23:58:20,500 (Finisher thread) - Plan: access=3DINDEX PRED WARN 2019-03-08T23:58:20,500 (Finisher thread) - Plan: join condition =3D [= index=3DI1549955498033 WARN 2019-03-08T23:58:20,500 (Finisher thread) - Plan: start conditions=3D[ WARN 2019-03-08T23:58:20,500 (Finisher thread) - Plan: EQUAL arg_left=3D[ C= OLUMN: PUBLIC.JOBS.STATUS WARN 2019-03-08T23:58:20,500 (Finisher thread) - Plan: ] arg_right=3D[ COLU= MN: C1 WARN 2019-03-08T23:58:20,500 (Finisher thread) - Plan: ]] WARN 2019-03-08T23:58:20,500 (Finisher thread) - Plan: end condition=3D[ WARN 2019-03-08T23:58:20,500 (Finisher thread) - Plan: EQUAL arg_left=3D[ C= OLUMN: PUBLIC.JOBS.STATUS WARN 2019-03-08T23:58:20,501 (Finisher thread) - Plan: ] arg_right=3D[ COLU= MN: C1 WARN 2019-03-08T23:58:20,501 (Finisher thread) - Plan: ]] WARN 2019-03-08T23:58:20,501 (Finisher thread) - Plan: other condition=3D[ WARN 2019-03-08T23:58:20,501 (Finisher thread) - Plan: EQUAL arg_left=3D[ C= OLUMN: PUBLIC.JOBS.STATUS WARN 2019-03-08T23:58:20,501 (Finisher thread) - Plan: ] arg_right=3D[ COLU= MN: C1 WARN 2019-03-08T23:58:20,501 (Finisher thread) - Plan: ]] WARN 2019-03-08T23:58:20,501 (Finisher thread) - Plan: ] WARN 2019-03-08T23:58:20,501 (Finisher thread) - Plan: ]] WARN 2019-03-08T23:58:20,501 (Finisher thread) - Plan: PARAMETERS=3D[ WARN 2019-03-08T23:58:20,501 (Finisher thread) - Plan: @0[DYNAMIC PARAM: , = TYPE =3D VARCHAR WARN 2019-03-08T23:58:20,501 (Finisher thread) - Plan: ] WARN 2019-03-08T23:58:20,501 (Finisher thread) - Plan: @1[DYNAMIC PARAM: , = TYPE =3D VARCHAR WARN 2019-03-08T23:58:20,501 (Finisher thread) - Plan: ] WARN 2019-03-08T23:58:20,501 (Finisher thread) - Plan: @2[DYNAMIC PARAM: , = TYPE =3D VARCHAR WARN 2019-03-08T23:58:20,501 (Finisher thread) - Plan: ]] WARN 2019-03-08T23:58:20,501 (Finisher thread) - Plan: SUBQUERIES[ WARN 2019-03-08T23:58:20,501 (Finisher thread) - Plan: [level=3D1 WARN 2019-03-08T23:58:20,502 (Finisher thread) - Plan: value expression]] WARN 2019-03-08T23:58:20,502 (Finisher thread) -=C2=A0 WARN 2019-03-08T23:58:20,504 (Delete startup thread) - Plan: isDistinctSele= ct=3D[false] WARN 2019-03-08T23:58:20,505 (Delete startup thread) - Plan: isGrouped=3D[f= alse] WARN 2019-03-08T23:58:20,505 (Delete startup thread) - Plan: isAggregated= =3D[false] WARN 2019-03-08T23:58:20,505 (Delete startup thread) - Plan: columns=3D[ CO= LUMN: PUBLIC.JOBS.ID not nullable WARN 2019-03-08T23:58:20,505 (Delete startup thread) - Plan:=C2=A0 WARN 2019-03-08T23:58:20,505 (Delete startup thread) - Plan: ] WARN 2019-03-08T23:58:20,505 (Delete startup thread) - Plan: [range variabl= e 1 WARN 2019-03-08T23:58:20,505 (Delete startup thread) - Plan: join type=3DIN= NER WARN 2019-03-08T23:58:20,505 (Delete startup thread) - Plan: table=3DJOBS WARN 2019-03-08T23:58:20,505 (Delete startup thread) - Plan: cardinality=3D= 3 WARN 2019-03-08T23:58:20,505 (Delete startup thread) - Plan: access=3DINDEX= PRED WARN 2019-03-08T23:58:20,505 (Delete startup thread) - Plan: join condition= =3D [index=3DI1549955498033 WARN 2019-03-08T23:58:20,505 (Delete startup thread) - Plan: start conditio= ns=3D[ WARN 2019-03-08T23:58:20,505 (Delete startup thread) - Plan: EQUAL arg_left= =3D[ COLUMN: PUBLIC.JOBS.STATUS WARN 2019-03-08T23:58:20,505 (Delete startup thread) - Plan: ] arg_right=3D= [ DYNAMIC PARAM: , TYPE =3D CHARACTER WARN 2019-03-08T23:58:20,507 (Delete startup thread) - Plan: ]] WARN 2019-03-08T23:58:20,507 (Document cleanup stuffer thread) - Plan: isDi= stinctSelect=3D[false] WARN 2019-03-08T23:58:20,507 (Document cleanup stuffer thread) - Plan: isGr= ouped=3D[false] WARN 2019-03-08T23:58:20,507 (Document cleanup stuffer thread) - Plan: isAg= gregated=3D[false] WARN 2019-03-08T23:58:20,507 (Document cleanup stuffer thread) - Plan: colu= mns=3D[ COLUMN: PUBLIC.JOBS.ID not nullable WARN 2019-03-08T23:58:20,507 (Document cleanup stuffer thread) - Plan:=C2= =A0 WARN 2019-03-08T23:58:20,507 (Document cleanup stuffer thread) - Plan: ] WARN 2019-03-08T23:58:20,507 (Document cleanup stuffer thread) - Plan: [ran= ge variable 1 WARN 2019-03-08T23:58:20,507 (Document cleanup stuffer thread) - Plan: join= type=3DINNER WARN 2019-03-08T23:58:20,507 (Document cleanup stuffer thread) - Plan: tabl= e=3DJOBS WARN 2019-03-08T23:58:20,507 (Document cleanup stuffer thread) - Plan: card= inality=3D3 WARN 2019-03-08T23:58:20,507 (Document cleanup stuffer thread) - Plan: acce= ss=3DINDEX PRED WARN 2019-03-08T23:58:20,507 (Document cleanup stuffer thread) - Plan: join= condition =3D [index=3DI1549955498033 WARN 2019-03-08T23:58:20,507 (Document cleanup stuffer thread) - Plan: star= t conditions=3D[ WARN 2019-03-08T23:58:20,507 (Document cleanup stuffer thread) - Plan: EQUA= L arg_left=3D[ COLUMN: PUBLIC.JOBS.STATUS WARN 2019-03-08T23:58:20,507 (Delete startup thread) - Plan: end condition= =3D[ WARN 2019-03-08T23:58:20,507 (Document cleanup stuffer thread) - Plan: ] ar= g_right=3D[ DYNAMIC PARAM: , TYPE =3D CHARACTER WARN 2019-03-08T23:58:20,507 (Document cleanup stuffer thread) - Plan: ]] WARN 2019-03-08T23:58:20,507 (Delete startup thread) - Plan: EQUAL arg_left= =3D[ COLUMN: PUBLIC.JOBS.STATUS WARN 2019-03-08T23:58:20,508 (Document cleanup stuffer thread) - Plan: end = condition=3D[ WARN 2019-03-08T23:58:20,508 (Delete startup thread) - Plan: ] arg_right=3D= [ DYNAMIC PARAM: , TYPE =3D CHARACTER WARN 2019-03-08T23:58:20,508 (Document cleanup stuffer thread) - Plan: EQUA= L arg_left=3D[ COLUMN: PUBLIC.JOBS.STATUS =C2=A0 Scenario-2 WEB: Illegal seed URL 'http://intranet.abcde.com/wps/wcm/connect/xxx/xxxweb= site/home > Under the right conditions, job aborts do not update "last checked" time > ------------------------------------------------------------------------ > > Key: CONNECTORS-880 > URL: https://issues.apache.org/jira/browse/CONNECTORS-880 > Project: ManifoldCF > Issue Type: Bug > Components: Framework crawler agent > Affects Versions: ManifoldCF 1.4.1 > Reporter: Karl Wright > Assignee: Karl Wright > Priority: Major > Fix For: ManifoldCF 1.6 > > > When a scheduled job is being considered to be started, MCF updates the l= ast-check field ONLY if the job didn't start. It relies on the job's compl= etion to set the last-check field in the case where the job does start. Bu= t if the job aborts, in at least one case the last-check field is NOT updat= ed. This leads to the job being run over and over again within the schedul= e window. -- This message was sent by Atlassian JIRA (v7.6.3#76005)