From user-return-5259-archive-asf-public=cust-asf.ponee.io@manifoldcf.apache.org Tue Jul 24 17:53:50 2018 Return-Path: X-Original-To: archive-asf-public@cust-asf.ponee.io Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by mx-eu-01.ponee.io (Postfix) with SMTP id 5CEFD180626 for ; Tue, 24 Jul 2018 17:53:48 +0200 (CEST) Received: (qmail 16262 invoked by uid 500); 24 Jul 2018 15:53:47 -0000 Mailing-List: contact user-help@manifoldcf.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@manifoldcf.apache.org Delivered-To: mailing list user@manifoldcf.apache.org Received: (qmail 16252 invoked by uid 99); 24 Jul 2018 15:53:47 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd2-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 24 Jul 2018 15:53:47 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd2-us-west.apache.org (ASF Mail Server at spamd2-us-west.apache.org) with ESMTP id B66BB1A1F07 for ; Tue, 24 Jul 2018 15:53:46 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd2-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: 3.091 X-Spam-Level: *** X-Spam-Status: No, score=3.091 tagged_above=-999 required=6.31 tests=[DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, HTML_MESSAGE=2, KAM_LINEPADDING=1.2, NORMAL_HTTP_TO_IP=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_PASS=-0.001, T_DKIMWL_WL_MED=-0.01, WEIRD_PORT=0.001] autolearn=disabled Authentication-Results: spamd2-us-west.apache.org (amavisd-new); dkim=pass (2048-bit key) header.d=gmail.com Received: from mx1-lw-us.apache.org ([10.40.0.8]) by localhost (spamd2-us-west.apache.org [10.40.0.9]) (amavisd-new, port 10024) with ESMTP id rGLeXS0SIfn2 for ; Tue, 24 Jul 2018 15:53:37 +0000 (UTC) Received: from mail-lf1-f43.google.com (mail-lf1-f43.google.com [209.85.167.43]) by mx1-lw-us.apache.org (ASF Mail Server at mx1-lw-us.apache.org) with ESMTPS id 4E59A5F2AC for ; Tue, 24 Jul 2018 15:53:36 +0000 (UTC) Received: by mail-lf1-f43.google.com with SMTP id n96-v6so3334431lfi.1 for ; Tue, 24 Jul 2018 08:53:36 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to; bh=QbPKXD/grrRaTFkG0Xfe/M0ZalW1m4shFOXfRyfiiHU=; b=FT5ongNlgFj6boGb6+AsnhHeAP5EBDXTNp7t1cWF11IA+uzELzTWf0d5f8bRVFeUnc eLuhqipctk7fsKzag0oEte0nQHv7X95yVdc9EVij03nNl7hRtOKIQFkptsfRaQYtpdV7 moBN19SoJidMh5YCKKyNKI8G4/ZDHobMcdTOP+xCn6/XPdQ7mbr1/OOiuqppICm+0G+V RAFQ3Ho43D1TVq5Osx6pnNptDct8QiyllMqFa42ohP4VaR3sXLyUFWaCOmPWneTsqiqr 3f45etNkoITfpnocA5sya+hUKIXZvOIXauhdUXWFFDqQw2JDUzUpkG4EhCDlStsU8bRT 6xkA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to; bh=QbPKXD/grrRaTFkG0Xfe/M0ZalW1m4shFOXfRyfiiHU=; b=CWKeIQsdBtkpZCSlWi1tEOz7GY4AkEeLdgi5nWEKVfWlGEpygVr68qcn0cV1UC9BsX QsV3eFwVFPYthldBvpcfX2su6i3NU+he3ganMm6PP94CW1x/tqTaTDlDht0PHIkJcO4j XZPIkF589b0kRyTfmtHjRXQrraHQqoX1sIF0Gg1GsaC0zjs0nsm32HpfJ7yRh73Vuy3d goTT3ljzEMBdf46XOYaFbkivQWExwrX3Ig318jzkJCRYhqycIDGQMDQmS18utoZYRBp+ ouuyebhZ818L/Mo0CbPUFi3pEOIrdy9zV8zXmWjkJEqsbrLu4BBnFbLfsTgOv5eTEX3W xkfA== X-Gm-Message-State: AOUpUlFeWS3lYxRhe7bmFkOAweU76ii9SWA/FRiiMQvs4XtweuHqwSpR 4xDF+aUyM7ZVtBgZHLeCVpqoEdmXDfTrpB6YYpf6zw== X-Google-Smtp-Source: AAOMgpfbxno5JkWEarN7N9ZLUjNVeC5+oerPm/uzqcIPhCBFyMiI02z25z5yZ3KB9kbhl9CYGDIfaouP/pzeMatRYyY= X-Received: by 2002:a19:cc0f:: with SMTP id c15-v6mr10050104lfg.145.1532447614690; Tue, 24 Jul 2018 08:53:34 -0700 (PDT) MIME-Version: 1.0 References: <025c01d42334$e4841540$ad8c3fc0$@citya.com> <02b101d4234f$fe1b5660$fa520320$@citya.com> <02cd01d4235b$2c97f7e0$85c7e7a0$@citya.com> <02f601d42364$0c14ab90$243e02b0$@citya.com> In-Reply-To: <02f601d42364$0c14ab90$243e02b0$@citya.com> From: Karl Wright Date: Tue, 24 Jul 2018 11:53:23 -0400 Message-ID: Subject: Re: Out of memory, one file bug i think To: user@manifoldcf.apache.org Content-Type: multipart/alternative; boundary="000000000000b5e9dd0571c0c3a6" --000000000000b5e9dd0571c0c3a6 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable The problem isn't with images in general; it's with certain kinds of images. There are optional dependencies in Tika for some kinds of images that we cannot include in the MCF distribution because of licensing problems. I don't know which kinds these are but apparently you are trying to index some of them. You will need to find and download the right jar and put it in the connector-common-lib folder for this to work. Karl On Tue, Jul 24, 2018 at 11:36 AM msaunier wrote: > On other crawl I extract images with sames parameters and I not have > problems with images. They are index without errors. Images are necessary > for this job. I try to recreate my job and test. > > > > Thanks, > > Maxence, > > > > > > > > > > *De :* Karl Wright [mailto:daddywri@gmail.com] > *Envoy=C3=A9 :* mardi 24 juillet 2018 17:32 > *=C3=80 :* user@manifoldcf.apache.org > *Objet :* Re: Out of memory, one file bug i think > > > > " java.lang.NoSuchMethodException: > org.openxmlformats.schemas.wordprocessingml.x2006.main.impl.CTPictureBase= Impl.(org.apache.xmlbeans.SchemaType, > boolean)" > > > > This exception is occurring because you are trying to extract content fro= m > an image. In order for this to work you need a jar that isn't supplied > with Tika for licensing reasons. Can you exclude images from your crawl? > > > > Karl > > > > > > On Tue, Jul 24, 2018 at 10:32 AM msaunier wrote: > > Hi Karl, > > > > With just connectors in debug I have that informations: > > > > [Thread-269948] INFO org.apache.zookeeper.ZooKeeper - Initiating client > connection, connectString=3Dkemp-formation-solr:2181 sessionTimeout=3D600= 00 > watcher=3Dorg.apache.solr.common.cloud.SolrZkClient$3@3c351b22 > > [Thread-269948-SendThread(kemp-formation-solr.citya.local:2181)] INFO > org.apache.zookeeper.ClientCnxn - Opening socket connection to server > kemp-formation-solr.citya.local/192.168.37.107:2181. Will not attempt to > authenticate using SASL (unknown error) > > [Thread-269948-SendThread(kemp-formation-solr.citya.local:2181)] INFO > org.apache.zookeeper.ClientCnxn - Socket connection established to > kemp-formation-solr.citya.local/192.168.37.107:2181, initiating session > > [Thread-269948-SendThread(kemp-formation-solr.citya.local:2181)] INFO > org.apache.zookeeper.ClientCnxn - Session establishment complete on serve= r > kemp-formation-solr.citya.local/192.168.37.107:2181, sessionid =3D > 0xff00000201970049, negotiated timeout =3D 40000 > > [Thread-269948] INFO org.apache.solr.common.cloud.ZkStateReader - Updated > live nodes from ZooKeeper... (0) -> (2) > > [Thread-269948] INFO > org.apache.solr.client.solrj.impl.ZkClientClusterStateProvider - Cluster = at > kemp-formation-solr:2181 ready > > java.lang.NoSuchMethodException: > org.openxmlformats.schemas.wordprocessingml.x2006.main.impl.CTPictureBase= Impl.(org.apache.xmlbeans.SchemaType, > boolean) > > at java.lang.Class.getConstructor0(Class.java:3082) > > at java.lang.Class.getDeclaredConstructor(Class.java:2178) > > at > org.apache.xmlbeans.impl.schema.SchemaTypeImpl.getJavaImplConstructor2(Sc= hemaTypeImpl.java:1817) > > at > org.apache.xmlbeans.impl.schema.SchemaTypeImpl.createUnattachedSubclass(S= chemaTypeImpl.java:1961) > > at > org.apache.xmlbeans.impl.schema.SchemaTypeImpl.createUnattachedNode(Schem= aTypeImpl.java:1950) > > at > org.apache.xmlbeans.impl.schema.SchemaTypeImpl.createElementType(SchemaTy= peImpl.java:1051) > > at > org.apache.xmlbeans.impl.values.XmlObjectBase.create_element_user(XmlObje= ctBase.java:938) > > at org.apache.xmlbeans.impl.store.Xobj.getUser(Xobj.java:1675) > > at org.apache.xmlbeans.impl.store.Cur.getUser(Cur.java:2659) > > at org.apache.xmlbeans.impl.store.Cur.getObject(Cur.java:2652) > > at > org.apache.xmlbeans.impl.store.Cursor._getObject(Cursor.java:995) > > at > org.apache.xmlbeans.impl.store.Cursor.getObject(Cursor.java:2904) > > at > org.apache.poi.xwpf.usermodel.XWPFDocument.onDocumentRead(XWPFDocument.ja= va:162) > > at org.apache.poi.POIXMLDocument.load(POIXMLDocument.java:169) > > at > org.apache.poi.xwpf.usermodel.XWPFDocument.(XWPFDocument.java:112) > > at > org.apache.poi.xwpf.extractor.XWPFWordExtractor.(XWPFWordExtractor.= java:60) > > at > org.apache.poi.extractor.ExtractorFactory.createExtractor(ExtractorFactor= y.java:243) > > at > org.apache.tika.parser.microsoft.ooxml.OOXMLExtractorFactory.parse(OOXMLE= xtractorFactory.java:105) > > at > org.apache.tika.parser.microsoft.ooxml.OOXMLParser.parse(OOXMLParser.java= :106) > > at > org.apache.tika.parser.CompositeParser.parse(CompositeParser.java:280) > > at > org.apache.tika.parser.CompositeParser.parse(CompositeParser.java:280) > > at > org.apache.tika.parser.AutoDetectParser.parse(AutoDetectParser.java:143) > > at > org.apache.manifoldcf.agents.transformation.tika.TikaParser.parse(TikaPar= ser.java:74) > > at > org.apache.manifoldcf.agents.transformation.tika.TikaExtractor.addOrRepla= ceDocumentWithException(TikaExtractor.java:235) > > at > org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester$Pipeli= neAddEntryPoint.addOrReplaceDocumentWithException(IncrementalIngester.java:= 3226) > > at > org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester$Pipeli= neAddFanout.sendDocument(IncrementalIngester.java:3077) > > at > org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester$Pipeli= neObjectWithVersions.addOrReplaceDocumentWithException(IncrementalIngester.= java:2708) > > at > org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester.docume= ntIngest(IncrementalIngester.java:756) > > at > org.apache.manifoldcf.crawler.system.WorkerThread$ProcessActivity.ingestD= ocumentWithException(WorkerThread.java:1583) > > at > org.apache.manifoldcf.crawler.system.WorkerThread$ProcessActivity.ingestD= ocumentWithException(WorkerThread.java:1548) > > at > org.apache.manifoldcf.crawler.connectors.sharedrive.SharedDriveConnector.= processDocuments(SharedDriveConnector.java:939) > > at > org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThread.java:3= 99) > > [Thread-35854-SendThread(kemp-formation-solr.citya.local:2181)] WARN > org.apache.zookeeper.ClientCnxn - Client session timed out, have not hear= d > from server in 28024ms for sessionid 0x100000050ae004d > > [Thread-35854-SendThread(kemp-formation-solr.citya.local:2181)] INFO > org.apache.zookeeper.ClientCnxn - Client session timed out, have not hear= d > from server in 28024ms for sessionid 0x100000050ae004d, closing socket > connection and attempting reconnect > > [zkCallback-16-thread-2] WARN > org.apache.solr.common.cloud.ConnectionManager - Watcher > org.apache.solr.common.cloud.ConnectionManager@5382340 name: > ZooKeeperConnection Watcher:kemp-formation-solr:2181 got event WatchedEve= nt > state:Disconnected type:None path:null path: null type: None > > [zkCallback-16-thread-2] WARN > org.apache.solr.common.cloud.ConnectionManager - zkClient has disconnecte= d > > [Thread-35854-SendThread(kemp-formation-solr.citya.local:2181)] INFO > org.apache.zookeeper.ClientCnxn - Opening socket connection to server > kemp-formation-solr.citya.local/192.168.37.107:2181. Will not attempt to > authenticate using SASL (unknown error) > > [Thread-35854-SendThread(kemp-formation-solr.citya.local:2181)] INFO > org.apache.zookeeper.ClientCnxn - Socket connection established to > kemp-formation-solr.citya.local/192.168.37.107:2181, initiating session > > agents process ran out of memory - shutting down > > java.lang.OutOfMemoryError: GC overhead limit exceeded > > at > org.apache.manifoldcf.core.database.Database.executeViaThread(Database.ja= va:737) > > at > org.apache.manifoldcf.core.database.Database.executeUncachedQuery(Databas= e.java:784) > > at > org.apache.manifoldcf.core.database.Database$QueryCacheExecutor.create(Da= tabase.java:1457) > > at > org.apache.manifoldcf.core.cachemanager.CacheManager.findObjectsAndExecut= e(CacheManager.java:146) > > at > org.apache.manifoldcf.core.database.Database.executeQuery(Database.java:2= 04) > > at > org.apache.manifoldcf.core.database.DBInterfacePostgreSQL.performQuery(DB= InterfacePostgreSQL.java:837) > > at > org.apache.manifoldcf.crawler.jobs.JobManager.getJobsReadyForInactivity(J= obManager.java:8024) > > at > org.apache.manifoldcf.crawler.system.JobNotificationThread.run(JobNotific= ationThread.java:76) > > agents process ran out of memory - shutting down > > java.lang.OutOfMemoryError: GC overhead limit exceeded > > at > org.postgresql.jdbc.PgConnection.prepareStatement(PgConnection.java:1200) > > at > org.postgresql.jdbc.PgConnection.prepareStatement(PgConnection.java:1583) > > at > org.postgresql.jdbc.PgConnection.prepareStatement(PgConnection.java:372) > > at > org.apache.manifoldcf.core.database.Database.execute(Database.java:896) > > at > org.apache.manifoldcf.core.database.Database$ExecuteQueryThread.run(Datab= ase.java:696) > > [Thread-35854-SendThread(kemp-formation-solr.citya.local:2181)] INFO > org.apache.zookeeper.ClientCnxn - Session establishment complete on serve= r > kemp-formation-solr.citya.local/192.168.37.107:2181, sessionid =3D > 0x100000050ae004d, negotiated timeout =3D 40000 > > [Thread-490] INFO org.eclipse.jetty.server.ServerConnector - Stopped > ServerConnector@2a640157{HTTP/1.1}{0.0.0.0:8345} > > agents process ran out of memory - shutting down > > java.lang.OutOfMemoryError: GC overhead limit exceeded > > at java.util.HashMap.resize(HashMap.java:704) > > at java.util.HashMap.putVal(HashMap.java:629) > > at java.util.HashMap.put(HashMap.java:612) > > at > org.apache.manifoldcf.core.cachemanager.CacheManager.findObjectsAndExecut= e(CacheManager.java:154) > > at > org.apache.manifoldcf.core.database.Database.executeQuery(Database.java:2= 04) > > at > org.apache.manifoldcf.core.database.DBInterfacePostgreSQL.performQuery(DB= InterfacePostgreSQL.java:837) > > at > org.apache.manifoldcf.crawler.jobs.JobManager.processParentHashSet(JobMan= ager.java:5642) > > at > org.apache.manifoldcf.crawler.jobs.JobManager.calculateAffectedRestoreCar= rydownChildren(JobManager.java:5581) > > at > org.apache.manifoldcf.crawler.jobs.JobManager.finishDocuments(JobManager.= java:5453) > > at > org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThread.java:5= 70) > > agents process ran out of memory - shutting down > > java.lang.OutOfMemoryError: GC overhead limit exceeded > > at java.util.Arrays.copyOf(Arrays.java:3308) > > at java.util.BitSet.ensureCapacity(BitSet.java:337) > > at java.util.BitSet.expandTo(BitSet.java:352) > > at java.util.BitSet.set(BitSet.java:447) > > at > de.l3s.boilerpipe.sax.BoilerpipeHTMLContentHandler.characters(BoilerpipeH= TMLContentHandler.java:267) > > at > org.apache.tika.parser.html.BoilerpipeContentHandler.characters(Boilerpip= eContentHandler.java:155) > > at > org.apache.tika.sax.ContentHandlerDecorator.characters(ContentHandlerDeco= rator.java:146) > > at > org.apache.tika.sax.SecureContentHandler.characters(SecureContentHandler.= java:270) > > at > org.apache.tika.sax.ContentHandlerDecorator.characters(ContentHandlerDeco= rator.java:146) > > at > org.apache.tika.sax.ContentHandlerDecorator.characters(ContentHandlerDeco= rator.java:146) > > at > org.apache.tika.sax.ContentHandlerDecorator.characters(ContentHandlerDeco= rator.java:146) > > at > org.apache.tika.sax.SafeContentHandler.access$001(SafeContentHandler.java= :46) > > at > org.apache.tika.sax.SafeContentHandler$1.write(SafeContentHandler.java:82= ) > > at > org.apache.tika.sax.SafeContentHandler.filter(SafeContentHandler.java:140= ) > > at > org.apache.tika.sax.SafeContentHandler.characters(SafeContentHandler.java= :287) > > at > org.apache.tika.sax.XHTMLContentHandler.characters(XHTMLContentHandler.ja= va:279) > > at > org.apache.tika.sax.XHTMLContentHandler.characters(XHTMLContentHandler.ja= va:306) > > at > org.apache.tika.parser.microsoft.ooxml.XSSFExcelExtractorDecorator$SheetT= extAsHTML.cell(XSSFExcelExtractorDecorator.java:431) > > at > org.apache.poi.xssf.eventusermodel.XSSFSheetXMLHandler.endElement(XSSFShe= etXMLHandler.java:380) > > at > org.apache.tika.parser.microsoft.ooxml.XSSFExcelExtractorDecorator$XSSFSh= eetInterestingPartsCapturer.endElement(XSSFExcelExtractorDecorator.java:520= ) > > at org.apache.xerces.parsers.AbstractSAXParser.endElement(Unknown > Source) > > at > org.apache.xerces.impl.XMLNSDocumentScannerImpl.scanEndElement(Unknown > Source) > > at > org.apache.xerces.impl.XMLDocumentFragmentScannerImpl$FragmentContentDisp= atcher.dispatch(Unknown > Source) > > at > org.apache.xerces.impl.XMLDocumentFragmentScannerImpl.scanDocument(Unknow= n > Source) > > at org.apache.xerces.parsers.XML11Configuration.parse(Unknown > Source) > > at org.apache.xerces.parsers.XML11Configuration.parse(Unknown > Source) > > at org.apache.xerces.parsers.XMLParser.parse(Unknown Source) > > at org.apache.xerces.parsers.AbstractSAXParser.parse(Unknown > Source) > > at > org.apache.xerces.jaxp.SAXParserImpl$JAXPSAXParser.parse(Unknown Source) > > at > org.apache.tika.parser.microsoft.ooxml.XSSFExcelExtractorDecorator.proces= sSheet(XSSFExcelExtractorDecorator.java:344) > > at > org.apache.tika.parser.microsoft.ooxml.XSSFExcelExtractorDecorator.buildX= HTML(XSSFExcelExtractorDecorator.java:167) > > at > org.apache.tika.parser.microsoft.ooxml.AbstractOOXMLExtractor.getXHTML(Ab= stractOOXMLExtractor.java:135) > > [Thread-490] INFO org.apache.zookeeper.ZooKeeper - Session: > 0x100000050ae004e closed > > [Thread-257943-EventThread] INFO org.apache.zookeeper.ClientCnxn - > EventThread shut down for session: 0x100000050ae004e > > [Thread-490] INFO org.apache.zookeeper.ZooKeeper - Session: > 0x100000050ae004d closed > > [Thread-35854-EventThread] INFO org.apache.zookeeper.ClientCnxn - > EventThread shut down for session: 0x100000050ae004d > > [Thread-490] INFO org.apache.zookeeper.ZooKeeper - Session: > 0x2000000b80d004a closed > > [Thread-8765-EventThread] INFO org.apache.zookeeper.ClientCnxn - > EventThread shut down for session: 0x2000000b80d004a > > [Thread-490] INFO org.apache.zookeeper.ZooKeeper - Session: > 0x2000000b80d004b closed > > [Thread-35853-EventThread] INFO org.apache.zookeeper.ClientCnxn - > EventThread shut down for session: 0x2000000b80d004b > > [Thread-490] INFO org.apache.zookeeper.ZooKeeper - Session: > 0xff00000201970046 closed > > [Thread-6991-EventThread] INFO org.apache.zookeeper.ClientCnxn - > EventThread shut down for session: 0xff00000201970046 > > [Thread-490] INFO org.apache.zookeeper.ZooKeeper - Session: > 0x100000050ae004c closed > > [Thread-8699-EventThread] INFO org.apache.zookeeper.ClientCnxn - > EventThread shut down for session: 0x100000050ae004c > > [Thread-490] INFO org.eclipse.jetty.server.handler.ContextHandler - > Stopped > o.e.j.w.WebAppContext@44d52de2{/mcf-api-service,file:/tmp/jetty-0.0.0.0-8= 345-mcf-api-service.war-_mcf-api-service-any-559052738855414857.dir/webapp/= ,UNAVAILABLE}{/opt/manifoldcf-trunk/bin/./../web-proprietary/war/mcf-api-se= rvice.war} > > [Thread-490] INFO org.eclipse.jetty.server.handler.ContextHandler - > Stopped > o.e.j.w.WebAppContext@60410cd{/mcf-authority-service,file:/tmp/jetty-0.0.= 0.0-8345-mcf-authority-service.war-_mcf-authority-service-any-9277703584113= 52606.dir/webapp/,UNAVAILABLE}{/opt/manifoldcf-trunk/bin/./../web-proprieta= ry/war/mcf-authority-service.war} > > [Thread-490] INFO org.apache.zookeeper.ZooKeeper - Session: > 0x2000000b80d004c closed > > [Thread-262666-EventThread] INFO org.apache.zookeeper.ClientCnxn - > EventThread shut down for session: 0x2000000b80d004c > > [Thread-490] INFO org.apache.zookeeper.ZooKeeper - Session: > 0xff00000201970048 closed > > [Thread-244171-EventThread] INFO org.apache.zookeeper.ClientCnxn - > EventThread shut down for session: 0xff00000201970048 > > [Thread-490] INFO org.apache.zookeeper.ZooKeeper - Session: > 0xff00000201970049 closed > > [Thread-269948-EventThread] INFO org.apache.zookeeper.ClientCnxn - > EventThread shut down for session: 0xff00000201970049 > > > > I have unactivate history to gain performances. So, can I find the last > file with SQL request? > > > > Maxence, > > > > *De :* Karl Wright [mailto:daddywri@gmail.com] > *Envoy=C3=A9 :* mardi 24 juillet 2018 16:04 > *=C3=80 :* user@manifoldcf.apache.org > *Objet :* Re: Out of memory, one file bug i think > > > > Hi Maxence, > > > > You would want to turn on connector debugging INSTEAD of the debugging > you've turned on, which is very noisy and not helpful. > > > > In global properties: org.apache.manifoldcf.connectors value DEBUG > > > > Karl > > > > > > On Tue, Jul 24, 2018 at 9:12 AM msaunier wrote: > > With debug: > > > > [Thread-5234-SendThread(kemp-formation-solr.citya.local:2181)] WARN > org.apache.zookeeper.ClientCnxn - Client session timed out, have not hear= d > from server in 28034ms for sessionid 0x100000050ae0049 > > [Thread-5234-SendThread(kemp-formation-solr.citya.local:2181)] INFO > org.apache.zookeeper.ClientCnxn - Client session timed out, have not hear= d > from server in 28034ms for sessionid 0x100000050ae0049, closing socket > connection and attempting reconnect > > [Thread-31532-SendThread(kemp-formation-solr.citya.local:2181)] WARN > org.apache.zookeeper.ClientCnxn - Client session timed out, have not hear= d > from server in 27708ms for sessionid 0xff00000201970044 > > [Thread-7573-SendThread(kemp-formation-solr.citya.local:2181)] WARN > org.apache.zookeeper.ClientCnxn - Client session timed out, have not hear= d > from server in 27737ms for sessionid 0xff00000201970043 > > [Thread-7573-SendThread(kemp-formation-solr.citya.local:2181)] INFO > org.apache.zookeeper.ClientCnxn - Client session timed out, have not hear= d > from server in 27737ms for sessionid 0xff00000201970043, closing socket > connection and attempting reconnect > > [Thread-31551-SendThread(kemp-formation-solr.citya.local:2181)] WARN > org.apache.zookeeper.ClientCnxn - Client session timed out, have not hear= d > from server in 28316ms for sessionid 0x100000050ae004b > > [Thread-7602-SendThread(kemp-formation-solr.citya.local:2181)] WARN > org.apache.zookeeper.ClientCnxn - Client session timed out, have not hear= d > from server in 28394ms for sessionid 0x2000000b80d0047 > > [Thread-7602-SendThread(kemp-formation-solr.citya.local:2181)] INFO > org.apache.zookeeper.ClientCnxn - Client session timed out, have not hear= d > from server in 28394ms for sessionid 0x2000000b80d0047, closing socket > connection and attempting reconnect > > [Thread-31532-SendThread(kemp-formation-solr.citya.local:2181)] INFO > org.apache.zookeeper.ClientCnxn - Client session timed out, have not hear= d > from server in 27708ms for sessionid 0xff00000201970044, closing socket > connection and attempting reconnect > > [Thread-5234-SendThread(kemp-formation-solr.citya.local:2181)] INFO > org.apache.zookeeper.ClientCnxn - Opening socket connection to server > kemp-formation-solr.citya.local/192.168.37.107:2181. Will not attempt to > authenticate using SASL (unknown error) > > agents process ran out of memory - shutting down > > [Thread-5234-SendThread(kemp-formation-solr.citya.local:2181)] INFO > org.apache.zookeeper.ClientCnxn - Socket connection established to > kemp-formation-solr.citya.local/192.168.37.107:2181, initiating session > > [Thread-7538-SendThread(kemp-formation-solr.citya.local:2181)] WARN > org.apache.zookeeper.ClientCnxn - Client session timed out, have not hear= d > from server in 36805ms for sessionid 0x2000000b80d0046 > > [Thread-7538-SendThread(kemp-formation-solr.citya.local:2181)] INFO > org.apache.zookeeper.ClientCnxn - Client session timed out, have not hear= d > from server in 36805ms for sessionid 0x2000000b80d0046, closing socket > connection and attempting reconnect > > java.lang.OutOfMemoryError: GC overhead limit exceeded > > at java.lang.StringBuilder.toString(StringBuilder.java:407) > > at > org.apache.manifoldcf.core.cachemanager.CacheManager.readSharedData(Cache= Manager.java:849) > > at > org.apache.manifoldcf.core.cachemanager.CacheManager.hasExpired(CacheMana= ger.java:483) > > at > org.apache.manifoldcf.core.cachemanager.CacheManager.lookupObject(CacheMa= nager.java:454) > > at > org.apache.manifoldcf.core.cachemanager.CacheManager.findObjectsAndExecut= e(CacheManager.java:131) > > at > org.apache.manifoldcf.core.database.Database.executeQuery(Database.java:2= 04) > > at > org.apache.manifoldcf.core.database.DBInterfacePostgreSQL.performQuery(DB= InterfacePostgreSQL.java:862) > > at > org.apache.manifoldcf.core.database.BaseTable.performQuery(BaseTable.java= :236) > > at > org.apache.manifoldcf.crawler.jobs.Jobs.deletingJobsPresent(Jobs.java:313= 3) > > at > org.apache.manifoldcf.crawler.jobs.JobManager.getNextDeletableDocuments(J= obManager.java:1862) > > at > org.apache.manifoldcf.crawler.system.DocumentDeleteStufferThread.run(Docu= mentDeleteStufferThread.java:108) > > [Thread-7573-SendThread(kemp-formation-solr.citya.local:2181)] INFO > org.apache.zookeeper.ClientCnxn - Opening socket connection to server > kemp-formation-solr.citya.local/192.168.37.107:2181. Will not attempt to > authenticate using SASL (unknown error) > > agents process ran out of memory - shutting down > > [Thread-7574-SendThread(kemp-formation-solr.citya.local:2181)] WARN > org.apache.zookeeper.ClientCnxn - Client session timed out, have not hear= d > from server in 27763ms for sessionid 0x100000050ae004a > > [Thread-7574-SendThread(kemp-formation-solr.citya.local:2181)] INFO > org.apache.zookeeper.ClientCnxn - Client session timed out, have not hear= d > from server in 27763ms for sessionid 0x100000050ae004a, closing socket > connection and attempting reconnect > > [zkCallback-3-thread-7] WARN > org.apache.solr.common.cloud.ConnectionManager - Watcher > org.apache.solr.common.cloud.ConnectionManager@7a5c701e name: > ZooKeeperConnection Watcher:kemp-formation-solr:2181 got event WatchedEve= nt > state:Disconnected type:None path:null path: null type: None > > [zkCallback-3-thread-7] WARN > org.apache.solr.common.cloud.ConnectionManager - zkClient has disconnecte= d > > [Thread-31551-SendThread(kemp-formation-solr.citya.local:2181)] INFO > org.apache.zookeeper.ClientCnxn - Client session timed out, have not hear= d > from server in 28316ms for sessionid 0x100000050ae004b, closing socket > connection and attempting reconnect > > java.lang.OutOfMemoryError: GC overhead limit exceeded > > [Thread-7573-SendThread(kemp-formation-solr.citya.local:2181)] INFO > org.apache.zookeeper.ClientCnxn - Socket connection established to > kemp-formation-solr.citya.local/192.168.37.107:2181, initiating session > > [zkCallback-11-thread-5] WARN > org.apache.solr.common.cloud.ConnectionManager - Watcher > org.apache.solr.common.cloud.ConnectionManager@53181a58 name: > ZooKeeperConnection Watcher:kemp-formation-solr:2181 got event WatchedEve= nt > state:Disconnected type:None path:null path: null type: None > > [zkCallback-11-thread-5] WARN > org.apache.solr.common.cloud.ConnectionManager - zkClient has disconnecte= d > > [Thread-7573-SendThread(kemp-formation-solr.citya.local:2181)] WARN > org.apache.zookeeper.ClientCnxn - Unable to reconnect to ZooKeeper servic= e, > session 0xff00000201970043 has expired > > [Thread-7573-SendThread(kemp-formation-solr.citya.local:2181)] INFO > org.apache.zookeeper.ClientCnxn - Unable to reconnect to ZooKeeper servic= e, > session 0xff00000201970043 has expired, closing socket connection > > [Thread-7573-EventThread] INFO org.apache.zookeeper.ClientCnxn - > EventThread shut down for session: 0xff00000201970043 > > [zkCallback-11-thread-2] WARN > org.apache.solr.common.cloud.ConnectionManager - Watcher > org.apache.solr.common.cloud.ConnectionManager@53181a58 name: > ZooKeeperConnection Watcher:kemp-formation-solr:2181 got event WatchedEve= nt > state:Expired type:None path:null path: null type: None > > [zkCallback-11-thread-2] WARN > org.apache.solr.common.cloud.ConnectionManager - Our previous ZooKeeper > session was expired. Attempting to reconnect to recover relationship with > ZooKeeper... > > [Thread-5234-SendThread(kemp-formation-solr.citya.local:2181)] WARN > org.apache.zookeeper.ClientCnxn - Unable to reconnect to ZooKeeper servic= e, > session 0x100000050ae0049 has expired > > [Thread-5234-SendThread(kemp-formation-solr.citya.local:2181)] INFO > org.apache.zookeeper.ClientCnxn - Unable to reconnect to ZooKeeper servic= e, > session 0x100000050ae0049 has expired, closing socket connection > > [zkCallback-11-thread-2] WARN > org.apache.solr.common.cloud.DefaultConnectionStrategy - Connection expir= ed > - starting a new one... > > [zkCallback-11-thread-2] INFO org.apache.zookeeper.ZooKeeper - Initiating > client connection, connectString=3Dkemp-formation-solr:2181 > sessionTimeout=3D60000 > watcher=3Dorg.apache.solr.common.cloud.ConnectionManager@53181a58 > > [Thread-5234-EventThread] INFO org.apache.zookeeper.ClientCnxn - > EventThread shut down for session: 0x100000050ae0049 > > [zkCallback-3-thread-4] WARN > org.apache.solr.common.cloud.ConnectionManager - Watcher > org.apache.solr.common.cloud.ConnectionManager@7a5c701e name: > ZooKeeperConnection Watcher:kemp-formation-solr:2181 got event WatchedEve= nt > state:Expired type:None path:null path: null type: None > > [zkCallback-3-thread-4] WARN > org.apache.solr.common.cloud.ConnectionManager - Our previous ZooKeeper > session was expired. Attempting to reconnect to recover relationship with > ZooKeeper... > > [zkCallback-3-thread-4] WARN > org.apache.solr.common.cloud.DefaultConnectionStrategy - Connection expir= ed > - starting a new one... > > [zkCallback-3-thread-4] INFO org.apache.zookeeper.ZooKeeper - Initiating > client connection, connectString=3Dkemp-formation-solr:2181 > sessionTimeout=3D60000 > watcher=3Dorg.apache.solr.common.cloud.ConnectionManager@7a5c701e > > [zkCallback-3-thread-4-SendThread(kemp-formation-solr.citya.local:2181)] > INFO org.apache.zookeeper.ClientCnxn - Opening socket connection to serve= r > kemp-formation-solr.citya.local/192.168.37.107:2181. Will not attempt to > authenticate using SASL (unknown error) > > [zkCallback-11-thread-2-SendThread(kemp-formation-solr.citya.local:2181)] > INFO org.apache.zookeeper.ClientCnxn - Opening socket connection to serve= r > kemp-formation-solr.citya.local/192.168.37.107:2181. Will not attempt to > authenticate using SASL (unknown error) > > [zkCallback-3-thread-4-SendThread(kemp-formation-solr.citya.local:2181)] > INFO org.apache.zookeeper.ClientCnxn - Socket connection established to > kemp-formation-solr.citya.local/192.168.37.107:2181, initiating session > > [zkCallback-11-thread-2-SendThread(kemp-formation-solr.citya.local:2181)] > INFO org.apache.zookeeper.ClientCnxn - Socket connection established to > kemp-formation-solr.citya.local/192.168.37.107:2181, initiating session > > [Thread-490] INFO org.eclipse.jetty.server.ServerConnector - Stopped > ServerConnector@2a640157{HTTP/1.1}{0.0.0.0:8345} > > [zkCallback-3-thread-4-SendThread(kemp-formation-solr.citya.local:2181)] > INFO org.apache.zookeeper.ClientCnxn - Session establishment complete on > server kemp-formation-solr.citya.local/192.168.37.107:2181, sessionid =3D > 0x2000000b80d0049, negotiated timeout =3D 40000 > > [zkCallback-11-thread-2-SendThread(kemp-formation-solr.citya.local:2181)] > INFO org.apache.zookeeper.ClientCnxn - Session establishment complete on > server kemp-formation-solr.citya.local/192.168.37.107:2181, sessionid =3D > 0xff00000201970045, negotiated timeout =3D 40000 > > agents process ran out of memory - shutting down > > java.lang.OutOfMemoryError: GC overhead limit exceeded > > agents process ran out of memory - shutting down > > java.lang.OutOfMemoryError: GC overhead limit exceeded > > at java.util.HashMap.newNode(HashMap.java:1747) > > at java.util.HashMap.putVal(HashMap.java:631) > > at java.util.HashMap.put(HashMap.java:612) > > at jcifs.util.transport.Transport.sendrecv(Transport.java:66) > > at jcifs.smb.SmbTransport.send(SmbTransport.java:661) > > at jcifs.smb.SmbSession.send(SmbSession.java:238) > > at jcifs.smb.SmbTree.send(SmbTree.java:119) > > at jcifs.smb.SmbFile.send(SmbFile.java:776) > > at > jcifs.smb.SmbFileInputStream.readDirect(SmbFileInputStream.java:181) > > at jcifs.smb.SmbFileInputStream.read(SmbFileInputStream.java:142) > > at > org.apache.manifoldcf.crawler.connectors.sharedrive.SharedDriveConnector.= processDocuments(SharedDriveConnector.java:903) > > at > org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThread.java:3= 99) > > [zkCallback-11-thread-2] INFO > org.apache.solr.common.cloud.ConnectionManager - Connection with ZooKeepe= r > reestablished. > > [zkCallback-3-thread-4] INFO > org.apache.solr.common.cloud.ConnectionManager - Connection with ZooKeepe= r > reestablished. > > agents process ran out of memory - shutting down > > java.lang.OutOfMemoryError: GC overhead limit exceeded > > [zkCallback-11-thread-2] INFO > org.apache.solr.common.cloud.DefaultConnectionStrategy - Reconnected to > ZooKeeper > > [zkCallback-11-thread-2] INFO > org.apache.solr.common.cloud.ConnectionManager - Connected:true > > [zkCallback-3-thread-4] INFO > org.apache.solr.common.cloud.DefaultConnectionStrategy - Reconnected to > ZooKeeper > > [zkCallback-3-thread-4] INFO > org.apache.solr.common.cloud.ConnectionManager - Connected:true > > [Thread-490] INFO org.apache.zookeeper.ZooKeeper - Session: > 0x2000000b80d0046 closed > > [zkCallback-21-thread-2] WARN > org.apache.solr.common.cloud.ConnectionManager - Watcher > org.apache.solr.common.cloud.ConnectionManager@381a7557 name: > ZooKeeperConnection Watcher:kemp-formation-solr:2181 got event WatchedEve= nt > state:Disconnected type:None path:null path: null type: None > > [zkCallback-21-thread-2] WARN > org.apache.solr.common.cloud.ConnectionManager - zkClient has disconnecte= d > > [Thread-7538-EventThread] INFO org.apache.zookeeper.ClientCnxn - > EventThread shut down for session: 0x2000000b80d0046 > > agents process ran out of memory - shutting down > > java.lang.OutOfMemoryError: GC overhead limit exceeded > > at java.util.regex.Matcher.(Matcher.java:225) > > at java.util.regex.Pattern.matcher(Pattern.java:1093) > > at > de.l3s.boilerpipe.util.UnicodeTokenizer.tokenize(UnicodeTokenizer.java:40= ) > > at > de.l3s.boilerpipe.sax.BoilerpipeHTMLContentHandler.flushBlock(BoilerpipeH= TMLContentHandler.java:296) > > at > de.l3s.boilerpipe.sax.BoilerpipeHTMLContentHandler.characters(BoilerpipeH= TMLContentHandler.java:198) > > at > org.apache.tika.parser.html.BoilerpipeContentHandler.characters(Boilerpip= eContentHandler.java:155) > > at > org.apache.tika.sax.ContentHandlerDecorator.characters(ContentHandlerDeco= rator.java:146) > > at > org.apache.tika.sax.SecureContentHandler.characters(SecureContentHandler.= java:270) > > at > org.apache.tika.sax.ContentHandlerDecorator.characters(ContentHandlerDeco= rator.java:146) > > at > org.apache.tika.sax.ContentHandlerDecorator.characters(ContentHandlerDeco= rator.java:146) > > at > org.apache.tika.sax.ContentHandlerDecorator.characters(ContentHandlerDeco= rator.java:146) > > at > org.apache.tika.sax.SafeContentHandler.access$001(SafeContentHandler.java= :46) > > at > org.apache.tika.sax.SafeContentHandler$1.write(SafeContentHandler.java:82= ) > > at > org.apache.tika.sax.SafeContentHandler.filter(SafeContentHandler.java:140= ) > > at > org.apache.tika.sax.SafeContentHandler.characters(SafeContentHandler.java= :287) > > at > org.apache.tika.sax.XHTMLContentHandler.characters(XHTMLContentHandler.ja= va:279) > > at > org.apache.tika.sax.ContentHandlerDecorator.characters(ContentHandlerDeco= rator.java:146) > > at > org.apache.tika.sax.xpath.MatchingContentHandler.characters(MatchingConte= ntHandler.java:85) > > at > org.apache.tika.sax.ContentHandlerDecorator.characters(ContentHandlerDeco= rator.java:146) > > at > org.apache.tika.sax.ContentHandlerDecorator.characters(ContentHandlerDeco= rator.java:146) > > at > org.apache.tika.sax.ContentHandlerDecorator.characters(ContentHandlerDeco= rator.java:146) > > at > org.apache.tika.sax.SecureContentHandler.characters(SecureContentHandler.= java:270) > > at > org.apache.tika.sax.ContentHandlerDecorator.characters(ContentHandlerDeco= rator.java:146) > > at > org.apache.tika.sax.ContentHandlerDecorator.characters(ContentHandlerDeco= rator.java:146) > > at > org.apache.tika.sax.ContentHandlerDecorator.characters(ContentHandlerDeco= rator.java:146) > > at > org.apache.tika.sax.SafeContentHandler.access$001(SafeContentHandler.java= :46) > > at > org.apache.tika.sax.SafeContentHandler$1.write(SafeContentHandler.java:82= ) > > at > org.apache.tika.sax.SafeContentHandler.filter(SafeContentHandler.java:140= ) > > at > org.apache.tika.sax.SafeContentHandler.characters(SafeContentHandler.java= :287) > > at > org.apache.tika.sax.XHTMLContentHandler.characters(XHTMLContentHandler.ja= va:279) > > at > org.apache.tika.sax.XHTMLContentHandler.characters(XHTMLContentHandler.ja= va:306) > > at > org.apache.tika.parser.microsoft.ooxml.XSSFExcelExtractorDecorator$SheetT= extAsHTML.cell(XSSFExcelExtractorDecorator.java:431) > > [zkCallback-19-thread-5] WARN > org.apache.solr.common.cloud.ConnectionManager - Watcher > org.apache.solr.common.cloud.ConnectionManager@43f7378f name: > ZooKeeperConnection Watcher:kemp-formation-solr:2181 got event WatchedEve= nt > state:Disconnected type:None path:null path: null type: None > > [zkCallback-19-thread-5] WARN > org.apache.solr.common.cloud.ConnectionManager - zkClient has disconnecte= d > > [zkCallback-15-thread-2] WARN > org.apache.solr.common.cloud.ConnectionManager - Watcher > org.apache.solr.common.cloud.ConnectionManager@6432608f name: > ZooKeeperConnection Watcher:kemp-formation-solr:2181 got event WatchedEve= nt > state:Disconnected type:None path:null path: null type: None > > [zkCallback-15-thread-2] WARN > org.apache.solr.common.cloud.ConnectionManager - zkClient has disconnecte= d > > [zkCallback-13-thread-3] WARN > org.apache.solr.common.cloud.ConnectionManager - Watcher > org.apache.solr.common.cloud.ConnectionManager@68bb3d74 name: > ZooKeeperConnection Watcher:kemp-formation-solr:2181 got event WatchedEve= nt > state:Disconnected type:None path:null path: null type: None > > [zkCallback-13-thread-3] WARN > org.apache.solr.common.cloud.ConnectionManager - zkClient has disconnecte= d > > agents process ran out of memory - shutting down > > java.lang.OutOfMemoryError: GC overhead limit exceeded > > at sun.nio.cs.UTF_8.newEncoder(UTF_8.java:72) > > at java.lang.StringCoding.encode(StringCoding.java:348) > > at java.lang.String.getBytes(String.java:941) > > at org.postgresql.core.Utils.encodeUTF8(Utils.java:53) > > at > org.postgresql.core.v3.QueryExecutorImpl.sendParse(QueryExecutorImpl.java= :1448) > > at > org.postgresql.core.v3.QueryExecutorImpl.sendOneQuery(QueryExecutorImpl.j= ava:1777) > > at > org.postgresql.core.v3.QueryExecutorImpl.sendQuery(QueryExecutorImpl.java= :1354) > > at > org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:2= 92) > > at > org.postgresql.jdbc.PgStatement.executeInternal(PgStatement.java:428) > > at org.postgresql.jdbc.PgStatement.execute(PgStatement.java:354) > > at > org.postgresql.jdbc.PgStatement.executeWithFlags(PgStatement.java:301) > > at > org.postgresql.jdbc.PgStatement.executeCachedSql(PgStatement.java:287) > > at > org.postgresql.jdbc.PgStatement.executeWithFlags(PgStatement.java:264) > > at org.postgresql.jdbc.PgStatement.execute(PgStatement.java:260) > > at > org.apache.manifoldcf.core.database.Database.execute(Database.java:876) > > at > org.apache.manifoldcf.core.database.Database$ExecuteQueryThread.run(Datab= ase.java:696) > > [Thread-490] INFO org.apache.zookeeper.ZooKeeper - Session: > 0xff00000201970044 closed > > [Thread-31532-EventThread] INFO org.apache.zookeeper.ClientCnxn - > EventThread shut down for session: 0xff00000201970044 > > [Thread-7574-SendThread(kemp-formation-solr.citya.local:2181)] INFO > org.apache.zookeeper.ClientCnxn - Opening socket connection to server > kemp-formation-solr.citya.local/192.168.37.107:2181. Will not attempt to > authenticate using SASL (unknown error) > > [Thread-7574-SendThread(kemp-formation-solr.citya.local:2181)] INFO > org.apache.zookeeper.ClientCnxn - Socket connection established to > kemp-formation-solr.citya.local/192.168.37.107:2181, initiating session > > [Thread-7574-SendThread(kemp-formation-solr.citya.local:2181)] INFO > org.apache.zookeeper.ClientCnxn - Session establishment complete on serve= r > kemp-formation-solr.citya.local/192.168.37.107:2181, sessionid =3D > 0x100000050ae004a, negotiated timeout =3D 40000 > > [Thread-490] INFO org.apache.zookeeper.ZooKeeper - Session: > 0x100000050ae004a closed > > [Thread-7574-EventThread] INFO org.apache.zookeeper.ClientCnxn - > EventThread shut down for session: 0x100000050ae004a > > [Thread-7602-SendThread(kemp-formation-solr.citya.local:2181)] INFO > org.apache.zookeeper.ClientCnxn - Opening socket connection to server > kemp-formation-solr.citya.local/192.168.37.107:2181. Will not attempt to > authenticate using SASL (unknown error) > > [Thread-7602-SendThread(kemp-formation-solr.citya.local:2181)] INFO > org.apache.zookeeper.ClientCnxn - Socket connection established to > kemp-formation-solr.citya.local/192.168.37.107:2181, initiating session > > [Thread-7602-SendThread(kemp-formation-solr.citya.local:2181)] INFO > org.apache.zookeeper.ClientCnxn - Session establishment complete on serve= r > kemp-formation-solr.citya.local/ > > --000000000000b5e9dd0571c0c3a6 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
The problem isn't with images in general; it's wit= h certain kinds of images.=C2=A0 There are optional dependencies in Tika fo= r some kinds of images that we cannot include in the MCF distribution becau= se of licensing problems.=C2=A0 I don't know which kinds these are but = apparently you are trying to index some of them.

You will need to fi= nd and download the right jar and put it in the connector-common-lib folder= for this to work.

Karl


On Tue, Jul 24, 2018 at 11:36 AM = msaunier <msaunier@citya.com&g= t; wrote:
On other crawl I extract images with sa= mes parameters and I not have problems with images. They are index without = errors. Images are necessary for this job. I try to recreate my job and tes= t.

=C2= =A0

Thanks,

Maxence,

=C2=A0

=C2=A0

=

=C2=A0

=C2=A0

De=C2=A0: Karl Wright [mailto:daddywri@gmail.com]
Envoy= =C3=A9=C2=A0: mardi 24 juillet 2018 17:32
=C3=80=C2=A0: user@manifoldcf.= apache.org
Objet=C2=A0: Re: Out of memory, one file bug i thi= nk

=C2=A0

<= div>

" java.lang.NoSuchMethodExce= ption: org.openxmlformats.schemas.wordprocessingml.x2006.main.impl.CTPictur= eBaseImpl.<init>(org.apache.xmlbeans.SchemaType, boolean)"

=C2=A0

This exception is occurring be= cause you are trying to extract content from an image.=C2=A0 In order for t= his to work you need a jar that isn't supplied with Tika for licensing = reasons.=C2=A0 Can you exclude images from your crawl?=

=C2=A0

Karl

=C2=A0

=C2=A0

On Tue, Ju= l 24, 2018 at 10:32 AM msaunier <msaunier@citya.com> wrote:

<= blockquote style=3D"border:none;border-left:solid #cccccc 1.0pt;padding:0cm= 0cm 0cm 6.0pt;margin-left:4.8pt;margin-right:0cm">

Hi Karl,

=C2=A0

<= span style=3D"font-size:11.0pt;font-family:"Calibri",sans-serif;c= olor:#1f497d">With just connectors in debug I have that informations:

=C2=A0

[Thread-269948] INFO org= .apache.zookeeper.ZooKeeper - Initiating client connection, connectString= =3Dkemp-formation-solr:2181 sessionTimeout=3D60000 watcher=3Dorg.apache.solr.common.cloud.SolrZkClient$3@3c351b22<= u>

[Thread-269948-SendT= hread(kemp-formation-solr.citya.local:2181)] INFO org.apache.zookeeper.Clie= ntCnxn - Opening socket connection to server kemp-formation-solr.citya.loca= l/192.168.37.107:2= 181. Will not attempt to authenticate using SASL (unknown error)=

[Thread-269948-Send= Thread(kemp-formation-solr.citya.local:2181)] INFO org.apache.zookeeper.Cli= entCnxn - Socket connection established to kemp-formation-solr.citya.local/= 192.168.37.107:218= 1, initiating session

[Thread-269948-SendThread(kemp-formation-solr.citya.local:2181= )] INFO org.apache.zookeeper.ClientCnxn - Session establishment complete on= server kemp-formation-solr.citya.local/192.168.37.107:2181, sessionid =3D 0xff0000020197= 0049, negotiated timeout =3D 40000

[Thread-269948] INFO org.apache.solr.common.cloud.ZkS= tateReader - Updated live nodes from ZooKeeper... (0) -> (2)

[Thread-269948] INFO org= .apache.solr.client.solrj.impl.ZkClientClusterStateProvider - Cluster at ke= mp-formation-solr:2181 ready

java.lang.NoSuchMethodException: org.openxmlformats.schemas= .wordprocessingml.x2006.main.impl.CTPictureBaseImpl.<init>(org.apache= .xmlbeans.SchemaType, boolean)

=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at java.lang.C= lass.getConstructor0(Class.java:3082)

=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at java= .lang.Class.getDeclaredConstructor(Class.java:2178)

=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0 at org.apache.xmlbeans.impl.schema.SchemaTypeImpl.getJavaImplConstru= ctor2(SchemaTypeImpl.java:1817)

=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at org.apache= .xmlbeans.impl.schema.SchemaTypeImpl.createUnattachedSubclass(SchemaTypeImp= l.java:1961)

= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at org.apache.xmlbeans.impl.sche= ma.SchemaTypeImpl.createUnattachedNode(SchemaTypeImpl.java:1950)<= /u>

=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0 at org.apache.xmlbeans.impl.schema.SchemaTypeImpl.cre= ateElementType(SchemaTypeImpl.java:1051)

=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at= org.apache.xmlbeans.impl.values.XmlObjectBase.create_element_user(XmlObjec= tBase.java:938)

=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at org.apache.xmlbeans.impl.= store.Xobj.getUser(Xobj.java:1675)

=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at org.apa= che.xmlbeans.impl.store.Cur.getUser(Cur.java:2659)

=

=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0 at org.apache.xmlbeans.impl.store.Cur.getObject(Cur.java:2652)

=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0 at org.apache.xmlbeans.impl.store.Cursor._getObjec= t(Cursor.java:995)

=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at org.apache.xmlbeans.imp= l.store.Cursor.getObject(Cursor.java:2904)

=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at= org.apache.poi.xwpf.usermodel.XWPFDocument.onDocumentRead(XWPFDocument.jav= a:162)

=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at org.apache.poi.POIXMLDocument.load(= POIXMLDocument.java:169)

=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at org.apache.poi.xw= pf.usermodel.XWPFDocument.<init>(XWPFDocument.java:112)=

=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0 at org.apache.poi.xwpf.extractor.XWPFWordExtractor.<i= nit>(XWPFWordExtractor.java:60)

=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at org.apa= che.poi.extractor.ExtractorFactory.createExtractor(ExtractorFactory.java:24= 3)

=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at org.apache.tika.parser.microsoft.ooxml= .OOXMLExtractorFactory.parse(OOXMLExtractorFactory.java:105)<= u>

=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0 at org.apache.tika.parser.microsoft.ooxml.OOXMLParser.parse= (OOXMLParser.java:106)

=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at org.apache.tika.par= ser.CompositeParser.parse(CompositeParser.java:280)

=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0 at org.apache.tika.parser.CompositeParser.parse(CompositeParser.java= :280)

=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at org.apache.tika.parser.AutoDetectPa= rser.parse(AutoDetectParser.java:143)

=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at org.= apache.manifoldcf.agents.transformation.tika.TikaParser.parse(TikaParser.ja= va:74)

=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at org.apache.manifoldcf.agents.transf= ormation.tika.TikaExtractor.addOrReplaceDocumentWithException(TikaExtractor= .java:235)

= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at org.apache.manifoldcf.agents.= incrementalingest.IncrementalIngester$PipelineAddEntryPoint.addOrReplaceDoc= umentWithException(IncrementalIngester.java:3226)

<= p class=3D"MsoNormal">=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0 at org.apache.manifoldcf.agents.incrementalingest.IncrementalIngeste= r$PipelineAddFanout.sendDocument(IncrementalIngester.java:3077)

=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0 at org.apache.manifoldcf.agents.incrementalingest.Increm= entalIngester$PipelineObjectWithVersions.addOrReplaceDocumentWithException(= IncrementalIngester.java:2708)

=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at org.apache.= manifoldcf.agents.incrementalingest.IncrementalIngester.documentIngest(Incr= ementalIngester.java:756)

=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at org.apache.manif= oldcf.crawler.system.WorkerThread$ProcessActivity.ingestDocumentWithExcepti= on(WorkerThread.java:1583)

<= span style=3D"font-size:11.0pt;font-family:"Calibri",sans-serif;c= olor:#1f497d">=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at org.apache.mani= foldcf.crawler.system.WorkerThread$ProcessActivity.ingestDocumentWithExcept= ion(WorkerThread.java:1548)

= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at org.apache.man= ifoldcf.crawler.connectors.sharedrive.SharedDriveConnector.processDocuments= (SharedDriveConnector.java:939)

=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at org.apache= .manifoldcf.crawler.system.WorkerThread.run(WorkerThread.java:399)

[Thread-35854-SendThr= ead(kemp-formation-solr.citya.local:2181)] WARN org.apache.zookeeper.Client= Cnxn - Client session timed out, have not heard from server in 28024ms for = sessionid 0x100000050ae004d

= [Thread-35854-SendThread(kemp-formation-solr.citya.local:218= 1)] INFO org.apache.zookeeper.ClientCnxn - Client session timed out, have n= ot heard from server in 28024ms for sessionid 0x100000050ae004d, closing so= cket connection and attempting reconnect

[zkCallback-16-thread-2] WARN org.apache.solr= .common.cloud.ConnectionManager - Watcher org.apache.solr.= common.cloud.ConnectionManager@5382340 name: ZooKeeperConnection Watche= r:kemp-formation-solr:2181 got event WatchedEvent state:Disconnected type:N= one path:null path: null type: None

[zkCallback-16-thread-2] WARN org.apache.solr.common= .cloud.ConnectionManager - zkClient has disconnected

[Thread-35854-SendThread(kemp-forma= tion-solr.citya.local:2181)] INFO org.apache.zookeeper.ClientCnxn - Opening= socket connection to server kemp-formation-solr.citya.local/192.168.37.107:2181. Will no= t attempt to authenticate using SASL (unknown error)

[Thread-35854-SendThread(kemp-forma= tion-solr.citya.local:2181)] INFO org.apache.zookeeper.ClientCnxn - Socket = connection established to kemp-formation-solr.citya.local/192.168.37.107:2181, initiating= session

agen= ts process ran out of memory - shutting down

java.lang.OutOfMemoryError: GC overhead lim= it exceeded

= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at org.apache.manifoldcf.core.da= tabase.Database.executeViaThread(Database.java:737)

=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0 at org.apache.manifoldcf.core.database.Database.executeUncachedQuery= (Database.java:784)

=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at org.apache.manifoldcf.= core.database.Database$QueryCacheExecutor.create(Database.java:1457)=

=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0 at org.apache.manifoldcf.core.cachemanager.CacheMa= nager.findObjectsAndExecute(CacheManager.java:146)

=

=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0 at org.apache.manifoldcf.core.database.Database.executeQuery(Databas= e.java:204)

= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at org.apache.manifoldcf.core.da= tabase.DBInterfacePostgreSQL.performQuery(DBInterfacePostgreSQL.java:837)

=C2=A0=C2=A0 = =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0at org.apache.manifoldcf.crawler.jobs.JobMana= ger.getJobsReadyForInactivity(JobManager.java:8024)

=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0 at org.apache.manifoldcf.crawler.system.JobNotificationThread.run(Jo= bNotificationThread.java:76)

agents process ran out of memory - shutting down<= /u>

java.lang.OutOfMemoryEr= ror: GC overhead limit exceeded

=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at org.postgr= esql.jdbc.PgConnection.prepareStatement(PgConnection.java:1200)

=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0 at org.postgresql.jdbc.PgConnection.prepareStatement(PgC= onnection.java:1583)

=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at org.postgresql.jdbc.P= gConnection.prepareStatement(PgConnection.java:372)

=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0 at org.apache.manifoldcf.core.database.Database.execute(Database.jav= a:896)

=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at org.apache.manifoldcf.core.database= .Database$ExecuteQueryThread.run(Database.java:696)

[Thread-35854-SendThread(kemp-format= ion-solr.citya.local:2181)] INFO org.apache.zookeeper.ClientCnxn - Session = establishment complete on server kemp-formation-solr.citya.local/192.168.37.107:2181, ses= sionid =3D 0x100000050ae004d, negotiated timeout =3D 40000=

[Thread-490] INFO org.eclipse= .jetty.server.ServerConnector - Stopped ServerConnector@2a640157{HTTP/1.1}{= 0.0.0.0:8345}<= u>

agents process ran o= ut of memory - shutting down

java.lang.OutOfMemoryError: GC overhead limit exceeded

=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0 at java.util.HashMap.resize(HashMap.java:704)

=C2=A0=C2=A0 = =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0at java.util.HashMap.putVal(HashMap.java:629)=

=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at java.util.HashMap.put(HashMap.java:612)

=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at org.apache.manifoldcf.core.cachemanager.C= acheManager.findObjectsAndExecute(CacheManager.java:154)

=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0 at org.apache.manifoldcf.core.database.Database.executeQuery(D= atabase.java:204)

=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at org.apache.manifoldcf.co= re.database.DBInterfacePostgreSQL.performQuery(DBInterfacePostgreSQL.java:8= 37)

=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at org.apache.manifoldcf.crawler.jobs.Job= Manager.processParentHashSet(JobManager.java:5642)

=

=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0 at org.apache.manifoldcf.crawler.jobs.JobManager.calculateAffectedRe= storeCarrydownChildren(JobManager.java:5581)

=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 = at org.apache.manifoldcf.crawler.jobs.JobManager.finishDocuments(JobManager= .java:5453)

= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at org.apache.manifoldcf.crawler= .system.WorkerThread.run(WorkerThread.java:570)

agents process ran out of memory - shutt= ing down

java= .lang.OutOfMemoryError: GC overhead limit exceeded

=

=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0 at java.util.Arrays.copyOf(Arrays.java:3308)

=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0 at java.util.BitSet.ensureCapacity(BitSet.java:337)=

=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0 at java.util.BitSet.expandTo(BitSet.java:352)=

=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0 at java.util.BitSet.set(BitSet.java:447)

=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0 at de.l3s.boilerpipe.sax.BoilerpipeHTMLContentHandler.chara= cters(BoilerpipeHTMLContentHandler.java:267)

=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 = at org.apache.tika.parser.html.BoilerpipeContentHandler.characters(Boilerpi= peContentHandler.java:155)

<= span style=3D"font-size:11.0pt;font-family:"Calibri",sans-serif;c= olor:#1f497d">=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at org.apache.tika= .sax.ContentHandlerDecorator.characters(ContentHandlerDecorator.java:146)

=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at org.apache.tika.sax.SecureContentHandler.= characters(SecureContentHandler.java:270)

=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at= org.apache.tika.sax.ContentHandlerDecorator.characters(ContentHandlerDecor= ator.java:146)

=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at org.apache.tika.sax.Conte= ntHandlerDecorator.characters(ContentHandlerDecorator.java:146)

=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0 at org.apache.tika.sax.ContentHandlerDecorator.character= s(ContentHandlerDecorator.java:146)

=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at org.ap= ache.tika.sax.SafeContentHandler.access$001(SafeContentHandler.java:46)

=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0 at org.apache.tika.sax.SafeContentHandler$1.wri= te(SafeContentHandler.java:82)

=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at org.apache.= tika.sax.SafeContentHandler.filter(SafeContentHandler.java:140)

=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0 at org.apache.tika.sax.SafeContentHandler.characters(Saf= eContentHandler.java:287)

=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at org.apache.tika.= sax.XHTMLContentHandler.characters(XHTMLContentHandler.java:279)<= /u>

=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0 at org.apache.tika.sax.XHTMLContentHandler.characters= (XHTMLContentHandler.java:306)

=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at org.apache.= tika.parser.microsoft.ooxml.XSSFExcelExtractorDecorator$SheetTextAsHTML.cel= l(XSSFExcelExtractorDecorator.java:431)

=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at or= g.apache.poi.xssf.eventusermodel.XSSFSheetXMLHandler.endElement(XSSFSheetXM= LHandler.java:380)

=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at org.apache.tika.parser.= microsoft.ooxml.XSSFExcelExtractorDecorator$XSSFSheetInterestingPartsCaptur= er.endElement(XSSFExcelExtractorDecorator.java:520)

=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0 at org.apache.xerces.parsers.AbstractSAXParser.endElement(Unknown So= urce)

=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at org.apache.xerces.impl.XMLNSDocumen= tScannerImpl.scanEndElement(Unknown Source)

=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 a= t org.apache.xerces.impl.XMLDocumentFragmentScannerImpl$FragmentContentDisp= atcher.dispatch(Unknown Source)

=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at org.apache= .xerces.impl.XMLDocumentFragmentScannerImpl.scanDocument(Unknown Source)

=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0 at org.apache.xerces.parsers.XML11Configuration= .parse(Unknown Source)

=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at org.apache.xerces.p= arsers.XML11Configuration.parse(Unknown Source)

=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0 at org.apache.xerces.parsers.XMLParser.parse(Unknown Source)<= /u>

=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0 at org.apache.xerces.parsers.AbstractSAXParser.parse(= Unknown Source)

=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at org.apache.xerces.jaxp.SA= XParserImpl$JAXPSAXParser.parse(Unknown Source)

=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0 at org.apache.tika.parser.microsoft.ooxml.XSSFExcelExtractorDecorator.p= rocessSheet(XSSFExcelExtractorDecorator.java:344)

<= p class=3D"MsoNormal">=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0 at org.apache.tika.parser.microsoft.ooxml.XSSFExcelExtractorDecorato= r.buildXHTML(XSSFExcelExtractorDecorator.java:167)

=

=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0 at org.apache.tika.parser.microsoft.ooxml.AbstractOOXMLExtractor.get= XHTML(AbstractOOXMLExtractor.java:135)

[Thread-490] INFO org.apache.zookeeper.ZooKeeper = - Session: 0x100000050ae004e closed

[Thread-257943-EventThread] INFO org.apache.zookeepe= r.ClientCnxn - EventThread shut down for session: 0x100000050ae004e<= u>

[Thread-490] INFO or= g.apache.zookeeper.ZooKeeper - Session: 0x100000050ae004d closed<= /u>

[Thread-35854-EventThre= ad] INFO org.apache.zookeeper.ClientCnxn - EventThread shut down for sessio= n: 0x100000050ae004d

[Thread-490] INFO org.apache.zookeeper.ZooKeeper - Session: 0x20000= 00b80d004a closed

[Thread-8765-EventThread] INFO org.apache.zookeeper.ClientCnxn - Event= Thread shut down for session: 0x2000000b80d004a

[Thread-490] INFO org.apache.zookeeper.Z= ooKeeper - Session: 0x2000000b80d004b closed

[Thread-35853-EventThread] INFO org.apache.= zookeeper.ClientCnxn - EventThread shut down for session: 0x2000000b80d004b=

[Thread-490]= INFO org.apache.zookeeper.ZooKeeper - Session: 0xff00000201970046 closed

[Thread-6991-E= ventThread] INFO org.apache.zookeeper.ClientCnxn - EventThread shut down fo= r session: 0xff00000201970046

[Thread-490] INFO org.apache.zookeeper.ZooKeeper - Session= : 0x100000050ae004c closed

<= span style=3D"font-size:11.0pt;font-family:"Calibri",sans-serif;c= olor:#1f497d">[Thread-8699-EventThread] INFO org.apache.zookeeper.ClientCnx= n - EventThread shut down for session: 0x100000050ae004c

[Thread-490] INFO org.eclipse.j= etty.server.handler.ContextHandler - Stopped o.e.j.w.WebAppContext@44d52de2{/mcf-api-service,fil= e:/tmp/jetty-0.0.0.0-8345-mcf-api-service.war-_mcf-api-service-any-55905273= 8855414857.dir/webapp/,UNAVAILABLE}{/opt/manifoldcf-trunk/bin/./../web-prop= rietary/war/mcf-api-service.war}

[Thread-490] INFO org.eclipse.jetty.server.handler.= ContextHandler - Stopped o.e.j.w.WebAppContext@60410cd{/mcf-authority-ser= vice,file:/tmp/jetty-0.0.0.0-8345-mcf-authority-service.war-_mcf-authority-= service-any-927770358411352606.dir/webapp/,UNAVAILABLE}{/opt/manifoldcf-tru= nk/bin/./../web-proprietary/war/mcf-authority-service.war}

[Thread-490] INFO org.apa= che.zookeeper.ZooKeeper - Session: 0x2000000b80d004c closed

[Thread-262666-EventThread] = INFO org.apache.zookeeper.ClientCnxn - EventThread shut down for session: 0= x2000000b80d004c

[Thread-490] INFO org.apache.zookeeper.ZooKeeper - Session: 0xff0000020= 1970048 closed

[Thread-244171-EventThread] INFO org.apache.zookeeper.ClientCnxn - Even= tThread shut down for session: 0xff00000201970048

<= p class=3D"MsoNormal">[Thread-490] INFO org.apache.zookeeper= .ZooKeeper - Session: 0xff00000201970049 closed

[Thread-269948-EventThread] INFO org.apa= che.zookeeper.ClientCnxn - EventThread shut down for session: 0xff000002019= 70049

=C2=A0<= /span>

I have unacti= vate history to gain performances. So, can I find the last file with SQL re= quest?

=C2=A0=

Maxence,

=C2=A0=

De=C2=A0: Karl Wrig= ht [mailto:daddywri= @gmail.com]
Envoy=C3=A9=C2=A0: mardi 24 juillet 2018 16:04=C3=80=C2=A0: user@manifoldcf.apache.org
Objet=C2=A0: Re: Out o= f memory, one file bug i think

=C2=A0

Hi Maxence,

=C2=A0

You would want to turn on connector debugging INSTEAD o= f the debugging you've turned on, which is very noisy and not helpful.<= u>

=C2=A0

=

In global properties: org.apache.manifold= cf.connectors value DEBUG

=C2=A0

Karl<= /u>

=C2=A0

=C2=A0

On Tue, Jul 24, 2018 at 9:12 AM msaunier <msaunier@citya.com> wrote:

With debug:

=C2=A0

[T= hread-5234-SendThread(kemp-formation-solr.citya.local:2181)] WARN org.apach= e.zookeeper.ClientCnxn - Client session timed out, have not heard from serv= er in 28034ms for sessionid 0x100000050ae0049

[Thread-5234-SendThread(kemp-formation-sol= r.citya.local:2181)] INFO org.apache.zookeeper.ClientCnxn - Client session = timed out, have not heard from server in 28034ms for sessionid 0x100000050a= e0049, closing socket connection and attempting reconnect<= /u>

[Thread-31532-SendThread(kemp-= formation-solr.citya.local:2181)] WARN org.apache.zookeeper.ClientCnxn - Cl= ient session timed out, have not heard from server in 27708ms for sessionid= 0xff00000201970044

[Thread-7573-SendThread(kemp-formation-solr.citya.local:2181)] WARN = org.apache.zookeeper.ClientCnxn - Client session timed out, have not heard = from server in 27737ms for sessionid 0xff00000201970043

[Thread-7573-SendThread(kemp-for= mation-solr.citya.local:2181)] INFO org.apache.zookeeper.ClientCnxn - Clien= t session timed out, have not heard from server in 27737ms for sessionid 0x= ff00000201970043, closing socket connection and attempting reconnect=

[Thread-31551-SendT= hread(kemp-formation-solr.citya.local:2181)] WARN org.apache.zookeeper.Clie= ntCnxn - Client session timed out, have not heard from server in 28316ms fo= r sessionid 0x100000050ae004b

[Thread-7602-SendThread(kemp-formation-solr.citya.local:21= 81)] WARN org.apache.zookeeper.ClientCnxn - Client session timed out, have = not heard from server in 28394ms for sessionid 0x2000000b80d0047<= /u>

[Thread-7602-SendThread= (kemp-formation-solr.citya.local:2181)] INFO org.apache.zookeeper.ClientCnx= n - Client session timed out, have not heard from server in 28394ms for ses= sionid 0x2000000b80d0047, closing socket connection and attempting reconnec= t

[Thread-315= 32-SendThread(kemp-formation-solr.citya.local:2181)] INFO org.apache.zookee= per.ClientCnxn - Client session timed out, have not heard from server in 27= 708ms for sessionid 0xff00000201970044, closing socket connection and attem= pting reconnect

[Thread-5234-SendThread(kemp-formation-solr.citya.local:2181)] INFO org= .apache.zookeeper.ClientCnxn - Opening socket connection to server kemp-for= mation-solr.citya.local/192.168.37.107:2181. Will not attempt to authenticate using SASL = (unknown error)

agents process ran out of memory - shutting down

[Thread-5234-SendThread(kemp-format= ion-solr.citya.local:2181)] INFO org.apache.zookeeper.ClientCnxn - Socket c= onnection established to kemp-formation-solr.citya.local/192.168.37.107:2181, initiating = session

[Thre= ad-7538-SendThread(kemp-formation-solr.citya.local:2181)] WARN org.apache.z= ookeeper.ClientCnxn - Client session timed out, have not heard from server = in 36805ms for sessionid 0x2000000b80d0046

[Thread-7538-SendThread(kemp-formation-solr.c= itya.local:2181)] INFO org.apache.zookeeper.ClientCnxn - Client session tim= ed out, have not heard from server in 36805ms for sessionid 0x2000000b80d00= 46, closing socket connection and attempting reconnect=

java.lang.OutOfMemoryError: GC ov= erhead limit exceeded

=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at java.lang.StringBuil= der.toString(StringBuilder.java:407)

=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at org.a= pache.manifoldcf.core.cachemanager.CacheManager.readSharedData(CacheManager= .java:849)

= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at org.apache.manifoldcf.core.ca= chemanager.CacheManager.hasExpired(CacheManager.java:483)<= /u>

=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0 at org.apache.manifoldcf.core.cachemanager.CacheManager.lookup= Object(CacheManager.java:454)

=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at org.apache.m= anifoldcf.core.cachemanager.CacheManager.findObjectsAndExecute(CacheManager= .java:131)

= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at org.apache.manifoldcf.core.da= tabase.Database.executeQuery(Database.java:204)

=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0 at org.apache.manifoldcf.core.database.DBInterfacePostgreSQL.performQue= ry(DBInterfacePostgreSQL.java:862)

=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at org.apa= che.manifoldcf.core.database.BaseTable.performQuery(BaseTable.java:236)

=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0 at org.apache.manifoldcf.crawler.jobs.Jobs.dele= tingJobsPresent(Jobs.java:3133)

=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at org.apache= .manifoldcf.crawler.jobs.JobManager.getNextDeletableDocuments(JobManager.ja= va:1862)

=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at org.apache.manifoldcf.crawler.sy= stem.DocumentDeleteStufferThread.run(DocumentDeleteStufferThread.java:108)<= /span>

[Thread-7573-= SendThread(kemp-formation-solr.citya.local:2181)] INFO org.apache.zookeeper= .ClientCnxn - Opening socket connection to server kemp-formation-solr.citya= .local/192.168.37.= 107:2181. Will not attempt to authenticate using SASL (unknown error)

agents process= ran out of memory - shutting down

[Thread-7574-SendThread(kemp-formation-solr.citya.loc= al:2181)] WARN org.apache.zookeeper.ClientCnxn - Client session timed out, = have not heard from server in 27763ms for sessionid 0x100000050ae004a

[Thread-7574-SendT= hread(kemp-formation-solr.citya.local:2181)] INFO org.apache.zookeeper.Clie= ntCnxn - Client session timed out, have not heard from server in 27763ms fo= r sessionid 0x100000050ae004a, closing socket connection and attempting rec= onnect

[zkCal= lback-3-thread-7] WARN org.apache.solr.common.cloud.ConnectionManager - Wat= cher org.apache.solr.common.cloud.ConnectionManager@7a5c7= 01e name: ZooKeeperConnection Watcher:kemp-formation-solr:2181 got even= t WatchedEvent state:Disconnected type:None path:null path: null type: None=

[zkCallback-= 3-thread-7] WARN org.apache.solr.common.cloud.ConnectionManager - zkClient = has disconnected

[Thread-31551-SendThread(kemp-formation-solr.citya.local:2181)] INFO or= g.apache.zookeeper.ClientCnxn - Client session timed out, have not heard fr= om server in 28316ms for sessionid 0x100000050ae004b, closing socket connec= tion and attempting reconnect

java.lang.OutOfMemoryError: GC overhead limit exceeded

[Thread-7573-Sen= dThread(kemp-formation-solr.citya.local:2181)] INFO org.apache.zookeeper.Cl= ientCnxn - Socket connection established to kemp-formation-solr.citya.local= /192.168.37.107:21= 81, initiating session

<= span style=3D"font-size:11.0pt;font-family:"Calibri",sans-serif;c= olor:#1f497d">[zkCallback-11-thread-5] WARN org.apache.solr.common.cloud.Co= nnectionManager - Watcher org.apache.solr.common.cloud.Co= nnectionManager@53181a58 name: ZooKeeperConnection Watcher:kemp-formati= on-solr:2181 got event WatchedEvent state:Disconnected type:None path:null = path: null type: None

[zkCallback-11-thread-5] WARN org.apache.solr.common.cloud.Connect= ionManager - zkClient has disconnected

[Thread-7573-SendThread(kemp-formation-solr.citya= .local:2181)] WARN org.apache.zookeeper.ClientCnxn - Unable to reconnect to= ZooKeeper service, session 0xff00000201970043 has expired=

[Thread-7573-SendThread(kemp-= formation-solr.citya.local:2181)] INFO org.apache.zookeeper.ClientCnxn - Un= able to reconnect to ZooKeeper service, session 0xff00000201970043 has expi= red, closing socket connection

[Thread-7573-EventThread] INFO org.apache.zookeeper.Clien= tCnxn - EventThread shut down for session: 0xff00000201970043=

[zkCallback-11-thread-2] W= ARN org.apache.solr.common.cloud.ConnectionManager - Watcher org.apache.solr.common.cloud.ConnectionManager@53181a58 name: ZooKe= eperConnection Watcher:kemp-formation-solr:2181 got event WatchedEvent stat= e:Expired type:None path:null path: null type: None

[zkCallback-11-thread-2] WARN org.ap= ache.solr.common.cloud.ConnectionManager - Our previous ZooKeeper session w= as expired. Attempting to reconnect to recover relationship with ZooKeeper.= ..

[Thread-52= 34-SendThread(kemp-formation-solr.citya.local:2181)] WARN org.apache.zookee= per.ClientCnxn - Unable to reconnect to ZooKeeper service, session 0x100000= 050ae0049 has expired

[Thread-5234-SendThread(kemp-formation-solr.citya.local:2181)] INF= O org.apache.zookeeper.ClientCnxn - Unable to reconnect to ZooKeeper servic= e, session 0x100000050ae0049 has expired, closing socket connection<= u>

[zkCallback-11-threa= d-2] WARN org.apache.solr.common.cloud.DefaultConnectionStrategy - Connecti= on expired - starting a new one...

[zkCallback-11-thread-2] INFO org.apache.zookeeper.Zo= oKeeper - Initiating client connection, connectString=3Dkemp-formation-solr= :2181 sessionTimeout=3D60000 watcher=3Dorg.apac= he.solr.common.cloud.ConnectionManager@53181a58

[Thread-5234-EventThread] INFO org.a= pache.zookeeper.ClientCnxn - EventThread shut down for session: 0x100000050= ae0049

[zkCal= lback-3-thread-4] WARN org.apache.solr.common.cloud.ConnectionManager - Wat= cher org.apache.solr.common.cloud.ConnectionManager@7a5c7= 01e name: ZooKeeperConnection Watcher:kemp-formation-solr:2181 got even= t WatchedEvent state:Expired type:None path:null path: null type: None

[zkCallback-3-thr= ead-4] WARN org.apache.solr.common.cloud.ConnectionManager - Our previous Z= ooKeeper session was expired. Attempting to reconnect to recover relationsh= ip with ZooKeeper...

[zkCallback-3-thread-4] WARN org.apache.solr.common.cloud.DefaultCo= nnectionStrategy - Connection expired - starting a new one...=

[zkCallback-3-thread-4] IN= FO org.apache.zookeeper.ZooKeeper - Initiating client connection, connectSt= ring=3Dkemp-formation-solr:2181 sessionTimeout=3D60000 watcher=3Dorg.apache.solr.common.cloud.ConnectionManager@7a5c701e

[zkCallbac= k-3-thread-4-SendThread(kemp-formation-solr.citya.local:2181)] INFO org.apa= che.zookeeper.ClientCnxn - Opening socket connection to server kemp-formati= on-solr.citya.local/192.168.37.107:2181. Will not attempt to authenticate using SASL (unk= nown error)

[= zkCallback-11-thread-2-SendThread(kemp-formation-solr.citya.local:2181)] IN= FO org.apache.zookeeper.ClientCnxn - Opening socket connection to server ke= mp-formation-solr.citya.local/192.168.37.107:2181. Will not attempt to authenticate using= SASL (unknown error)

[zkCallback-3-thread-4-SendThread(kemp-formation-solr.citya.local:= 2181)] INFO org.apache.zookeeper.ClientCnxn - Socket connection established= to kemp-formation-solr.citya.local/192.168.37.107:2181, initiating session=

[zkCallback-11-thread-2-Se= ndThread(kemp-formation-solr.citya.local:2181)] INFO org.apache.zookeeper.C= lientCnxn - Socket connection established to kemp-formation-solr.citya.loca= l/192.168.37.107:2= 181, initiating session

= [Thread-490] INFO org.eclipse.jetty.server.ServerConnector -= Stopped ServerConnector@2a640157{HTTP/1.1}{0.0.0.0:8345}

[zkCallback-3-thread-4-SendThread(kemp-formation-so= lr.citya.local:2181)] INFO org.apache.zookeeper.ClientCnxn - Session establ= ishment complete on server kemp-formation-solr.citya.local/192.168.37.107:2181, sessionid= =3D 0x2000000b80d0049, negotiated timeout =3D 40000

[zkCallback-11-thread-2-SendThread(= kemp-formation-solr.citya.local:2181)] INFO org.apache.zookeeper.ClientCnxn= - Session establishment complete on server kemp-formation-solr.citya.local= /192.168.37.107:21= 81, sessionid =3D 0xff00000201970045, negotiated timeout =3D 40000

agents process ra= n out of memory - shutting down

java.lang.OutOfMemoryError: GC overhead limit exceeded

agents process= ran out of memory - shutting down

java.lang.OutOfMemoryError: GC overhead limit exceede= d

=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at java.util.HashMap.newNode(HashMap.java= :1747)

=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at java.util.HashMap.putVal(HashMap.ja= va:631)

=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at java.util.HashMap.put(HashMap.ja= va:612)

=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at jcifs.util.transport.Transport.s= endrecv(Transport.java:66)

<= span style=3D"font-size:11.0pt;font-family:"Calibri",sans-serif;c= olor:#1f497d">=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at jcifs.smb.SmbTr= ansport.send(SmbTransport.java:661)

=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at jcifs.= smb.SmbSession.send(SmbSession.java:238)

=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at= jcifs.smb.SmbTree.send(SmbTree.java:119)

=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at= jcifs.smb.SmbFile.send(SmbFile.java:776)

=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at= jcifs.smb.SmbFileInputStream.readDirect(SmbFileInputStream.java:181)

=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0 at jcifs.smb.SmbFileInputStream.read(SmbFileInputS= tream.java:142)

=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at org.apache.manifoldcf.cra= wler.connectors.sharedrive.SharedDriveConnector.processDocuments(SharedDriv= eConnector.java:903)

=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at org.apache.manifoldcf= .crawler.system.WorkerThread.run(WorkerThread.java:399)

[zkCallback-11-thread-2] INFO or= g.apache.solr.common.cloud.ConnectionManager - Connection with ZooKeeper re= established.

= [zkCallback-3-thread-4] INFO org.apache.solr.common.cloud.ConnectionManager= - Connection with ZooKeeper reestablished.

agents process ran out of memory - shutting = down

java.lan= g.OutOfMemoryError: GC overhead limit exceeded

[zkCallback-11-thread-2] INFO org.apache.= solr.common.cloud.DefaultConnectionStrategy - Reconnected to ZooKeeper

[zkCallback-11-th= read-2] INFO org.apache.solr.common.cloud.ConnectionManager - Connected:tru= e

[zkCallback= -3-thread-4] INFO org.apache.solr.common.cloud.DefaultConnectionStrategy - = Reconnected to ZooKeeper

[zkCallback-3-thread-4] INFO org.apache.solr.common.cloud.Conne= ctionManager - Connected:true

[Thread-490] INFO org.apache.zookeeper.ZooKeeper - Session= : 0x2000000b80d0046 closed

<= span style=3D"font-size:11.0pt;font-family:"Calibri",sans-serif;c= olor:#1f497d">[zkCallback-21-thread-2] WARN org.apache.solr.common.cloud.Co= nnectionManager - Watcher org.apache.solr.common.cloud.Co= nnectionManager@381a7557 name: ZooKeeperConnection Watcher:kemp-formati= on-solr:2181 got event WatchedEvent state:Disconnected type:None path:null = path: null type: None

[zkCallback-21-thread-2] WARN org.apache.solr.common.cloud.Connect= ionManager - zkClient has disconnected

[Thread-7538-EventThread] INFO org.apache.zookeep= er.ClientCnxn - EventThread shut down for session: 0x2000000b80d0046=

agents process ran = out of memory - shutting down

java.lang.OutOfMemoryError: GC overhead limit exceeded

=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0 at java.util.regex.Matcher.<init>(Matcher= .java:225)

= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at java.util.regex.Pattern.match= er(Pattern.java:1093)

=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at de.l3s.boilerpipe.ut= il.UnicodeTokenizer.tokenize(UnicodeTokenizer.java:40)=

=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0 at de.l3s.boilerpipe.sax.BoilerpipeHTMLContentHandler.flushBlock(= BoilerpipeHTMLContentHandler.java:296)

=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at de.= l3s.boilerpipe.sax.BoilerpipeHTMLContentHandler.characters(BoilerpipeHTMLCo= ntentHandler.java:198)

=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at org.apache.tika.par= ser.html.BoilerpipeContentHandler.characters(BoilerpipeContentHandler.java:= 155)

=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at org.apache.tika.sax.ContentHandlerD= ecorator.characters(ContentHandlerDecorator.java:146)<= /p>

=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0 at org.apache.tika.sax.SecureContentHandler.characters(SecureCont= entHandler.java:270)

=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at org.apache.tika.sax.C= ontentHandlerDecorator.characters(ContentHandlerDecorator.java:146)<= u>

=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0 at org.apache.tika.sax.ContentHandlerDecorator.cha= racters(ContentHandlerDecorator.java:146)

=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at= org.apache.tika.sax.ContentHandlerDecorator.characters(ContentHandlerDecor= ator.java:146)

=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at org.apache.tika.sax.SafeC= ontentHandler.access$001(SafeContentHandler.java:46)

=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0 at org.apache.tika.sax.SafeContentHandler$1.write(SafeContentHand= ler.java:82)

= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at org.apache.tika.sax.SafeConte= ntHandler.filter(SafeContentHandler.java:140)

=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= at org.apache.tika.sax.SafeContentHandler.characters(SafeContentHandler.ja= va:287)

=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at org.apache.tika.sax.XHTMLContent= Handler.characters(XHTMLContentHandler.java:279)

=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0 at org.apache.tika.sax.ContentHandlerDecorator.characters(ContentHandle= rDecorator.java:146)

=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at org.apache.tika.sax.x= path.MatchingContentHandler.characters(MatchingContentHandler.java:85)

=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0 at org.apache.tika.sax.ContentHandlerDecorator.= characters(ContentHandlerDecorator.java:146)

=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 = at org.apache.tika.sax.ContentHandlerDecorator.characters(ContentHandlerDec= orator.java:146)

=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at org.apache.tika.sax.Conte= ntHandlerDecorator.characters(ContentHandlerDecorator.java:146)

=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0 at org.apache.tika.sax.SecureContentHandler.characters(S= ecureContentHandler.java:270)

=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at org.apache.t= ika.sax.ContentHandlerDecorator.characters(ContentHandlerDecorator.java:146= )

=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at org.apache.tika.sax.ContentHandlerDeco= rator.characters(ContentHandlerDecorator.java:146)

=

=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0 at org.apache.tika.sax.ContentHandlerDecorator.characters(ContentHan= dlerDecorator.java:146)

=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at org.apache.tika.sa= x.SafeContentHandler.access$001(SafeContentHandler.java:46)

=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0 at org.apache.tika.sax.SafeContentHandler$1.write(SafeConte= ntHandler.java:82)

=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at org.apache.tika.sax.Saf= eContentHandler.filter(SafeContentHandler.java:140)

=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0 at org.apache.tika.sax.SafeContentHandler.characters(SafeContentHand= ler.java:287)

=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at org.apache.tika.sax.XHTMLCon= tentHandler.characters(XHTMLContentHandler.java:279)

=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0 at org.apache.tika.sax.XHTMLContentHandler.characters(XHTMLConten= tHandler.java:306)

=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at org.apache.tika.parser.= microsoft.ooxml.XSSFExcelExtractorDecorator$SheetTextAsHTML.cell(XSSFExcelE= xtractorDecorator.java:431)

= [zkCallback-19-thread-5] WARN org.apache.solr.common.cloud.C= onnectionManager - Watcher org.apache.solr.common.cloud.C= onnectionManager@43f7378f name: ZooKeeperConnection Watcher:kemp-format= ion-solr:2181 got event WatchedEvent state:Disconnected type:None path:null= path: null type: None

[zkCallback-19-thread-5] WARN org.apache.solr.common.cloud.Connec= tionManager - zkClient has disconnected

[zkCallback-15-thread-2] WARN org.apache.solr.co= mmon.cloud.ConnectionManager - Watcher org.apache.solr.co= mmon.cloud.ConnectionManager@6432608f name: ZooKeeperConnection Watcher= :kemp-formation-solr:2181 got event WatchedEvent state:Disconnected type:No= ne path:null path: null type: None

[zkCallback-15-thread-2] WARN org.apache.solr.common.= cloud.ConnectionManager - zkClient has disconnected

[zkCallback-13-thread-3] WARN org.ap= ache.solr.common.cloud.ConnectionManager - Watcher org.ap= ache.solr.common.cloud.ConnectionManager@68bb3d74 name: ZooKeeperConnec= tion Watcher:kemp-formation-solr:2181 got event WatchedEvent state:Disconne= cted type:None path:null path: null type: None

[zkCallback-13-thread-3] WARN org.apache.= solr.common.cloud.ConnectionManager - zkClient has disconnected

agents process ran out o= f memory - shutting down

java.lang.OutOfMemoryError: GC overhead limit exceeded

=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0 at sun.nio.cs.UTF_8.newEncoder(UTF_8.java:72)<= u>

=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0 at java.lang.StringCoding.encode(StringCoding.java= :348)

=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at java.lang.String.getBytes(String.ja= va:941)

=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at org.postgresql.core.Utils.encode= UTF8(Utils.java:53)

=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at org.postgresql.core.v3= .QueryExecutorImpl.sendParse(QueryExecutorImpl.java:1448)<= /u>

=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0 at org.postgresql.core.v3.QueryExecutorImpl.sendOneQuery(Query= ExecutorImpl.java:1777)

=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at org.postgresql.cor= e.v3.QueryExecutorImpl.sendQuery(QueryExecutorImpl.java:1354)=

=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0 at org.postgresql.core.v3.QueryExecutorImpl.execute(Quer= yExecutorImpl.java:292)

=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at org.postgresql.jdb= c.PgStatement.executeInternal(PgStatement.java:428)

=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0 at org.postgresql.jdbc.PgStatement.execute(PgStatement.java:354)

=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0 at org.postgresql.jdbc.PgStatement.executeWithF= lags(PgStatement.java:301)

<= span style=3D"font-size:11.0pt;font-family:"Calibri",sans-serif;c= olor:#1f497d">=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at org.postgresql.= jdbc.PgStatement.executeCachedSql(PgStatement.java:287)

=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0 at org.postgresql.jdbc.PgStatement.executeWithFlags(PgStatemen= t.java:264)

= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at org.postgresql.jdbc.PgStateme= nt.execute(PgStatement.java:260)

=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at org.apach= e.manifoldcf.core.database.Database.execute(Database.java:876)

=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0 at org.apache.manifoldcf.core.database.Database$ExecuteQ= ueryThread.run(Database.java:696)

[Thread-490] INFO org.apache.zookeeper.ZooKeeper - Ses= sion: 0xff00000201970044 closed

[Thread-31532-EventThread] INFO org.apache.zookeeper.Cli= entCnxn - EventThread shut down for session: 0xff00000201970044

[Thread-7574-SendThread(= kemp-formation-solr.citya.local:2181)] INFO org.apache.zookeeper.ClientCnxn= - Opening socket connection to server kemp-formation-solr.citya.local/192.168.37.107:2181. Will not attempt to authenticate using SASL (unknown error)

[Thread-7574-SendThread(k= emp-formation-solr.citya.local:2181)] INFO org.apache.zookeeper.ClientCnxn = - Socket connection established to kemp-formation-solr.citya.local/192.168.37.107:2181, = initiating session

[Thread-7574-SendThread(kemp-formation-solr.citya.local:2181)] INFO o= rg.apache.zookeeper.ClientCnxn - Session establishment complete on server k= emp-formation-solr.citya.local/192.168.37.107:2181, sessionid =3D 0x100000050ae004a, nego= tiated timeout =3D 40000

[Thread-490] INFO org.apache.zookeeper.ZooKeeper - Session: 0x1= 00000050ae004a closed

[Thread-7574-EventThread] INFO org.apache.zookeeper.ClientCnxn - E= ventThread shut down for session: 0x100000050ae004a

[Thread-7602-SendThread(kemp-formati= on-solr.citya.local:2181)] INFO org.apache.zookeeper.ClientCnxn - Opening s= ocket connection to server kemp-formation-solr.citya.local/192.168.37.107:2181. Will not = attempt to authenticate using SASL (unknown error)

=

[Thread-7602-SendThread(kemp-formatio= n-solr.citya.local:2181)] INFO org.apache.zookeeper.ClientCnxn - Socket con= nection established to kemp-formation-solr.citya.local/192.168.37.107:2181, initiating se= ssion

[Thread= -7602-SendThread(kemp-formation-solr.citya.local:2181)] INFO org.apache.zoo= keeper.ClientCnxn - Session establishment complete on server kemp-formation= -solr.citya.local/=

--000000000000b5e9dd0571c0c3a6--