manifoldcf-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Karl Wright (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (CONNECTORS-1116) "Too many open files" IO exception from Rest API
Date Wed, 26 Nov 2014 14:22:12 GMT

    [ https://issues.apache.org/jira/browse/CONNECTORS-1116?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14226217#comment-14226217
] 

Karl Wright commented on CONNECTORS-1116:
-----------------------------------------

What is the exact Rest API call that you are doing to get this to
happen?

Karl

Sent from my Windows Phone
From: Adrian Conlon (JIRA)
Sent: 11/26/2014 8:00 AM
To: dev@manifoldcf.apache.org
Subject: [jira] [Created] (CONNECTORS-1116) "Too many open files" IO
exception from Rest API
Adrian Conlon created CONNECTORS-1116:
-----------------------------------------

             Summary: "Too many open files" IO exception from Rest API
                 Key: CONNECTORS-1116
                 URL: https://issues.apache.org/jira/browse/CONNECTORS-1116
             Project: ManifoldCF
          Issue Type: Bug
          Components: API
    Affects Versions: ManifoldCF 1.7.2
         Environment: Ubuntu Linux, Tomcat, Zookeeper
            Reporter: Adrian Conlon


Using the REST API to get a list of all the job definitions.

Using lsof, I can see that the total number of open files jumps from
around 160 to 4500 as the call is made.

Also, I should note that there are around 4000 jobs under MCF control.

Exception below:

ERROR 2014-11-26 12:46:07,042 (http-bio-8080-exec-9) - Zookeeper
initialization error: Couldn't instantiate
org.apache.zookeeper.ClientCnxnSocketNIO
org.apache.manifoldcf.core.interfaces.ManifoldCFException: Zookeeper
initialization error: Couldn't instantiate
org.apache.zookeeper.ClientCnxnSocketNIO
        at org.apache.manifoldcf.core.lockmanager.ZooKeeperConnection.createSession(ZooKeeperConnection.java:82)
        at org.apache.manifoldcf.core.lockmanager.ZooKeeperConnection.<init>(ZooKeeperConnection.java:66)
        at org.apache.manifoldcf.core.lockmanager.ZooKeeperConnectionPool.grab(ZooKeeperConnectionPool.java:48)
        at org.apache.manifoldcf.core.lockmanager.ZooKeeperLockObject.obtainGlobalReadLock(ZooKeeperLockObject.java:190)
        at org.apache.manifoldcf.core.lockmanager.LockObject.enterReadLock(LockObject.java:310)
        at org.apache.manifoldcf.core.lockmanager.LockGate.enterReadLock(LockGate.java:261)
        at org.apache.manifoldcf.core.lockmanager.BaseLockManager.enter(BaseLockManager.java:1522)
        at org.apache.manifoldcf.core.lockmanager.BaseLockManager.enterLocks(BaseLockManager.java:803)
        at org.apache.manifoldcf.core.cachemanager.CacheManager.enterCache(CacheManager.java:299)
        at org.apache.manifoldcf.core.cachemanager.CacheManager.findObjectsAndExecute(CacheManager.java:100)
        at org.apache.manifoldcf.crawler.jobs.Jobs.loadMultiple(Jobs.java:926)
        at org.apache.manifoldcf.crawler.jobs.Jobs.getAll(Jobs.java:774)
        at org.apache.manifoldcf.crawler.jobs.JobManager.getAllJobs(JobManager.java:760)
        at org.apache.manifoldcf.crawler.system.ManifoldCF.apiReadJobs(ManifoldCF.java:1299)
        at org.apache.manifoldcf.crawler.system.ManifoldCF.executeReadCommand(ManifoldCF.java:2919)
        at org.apache.manifoldcf.apiservlet.APIServlet.executeRead(APIServlet.java:232)
        at org.apache.manifoldcf.apiservlet.APIServlet.doGet(APIServlet.java:78)
        at javax.servlet.http.HttpServlet.service(HttpServlet.java:620)
        at javax.servlet.http.HttpServlet.service(HttpServlet.java:727)
        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:303)
        at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
        at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:220)
        at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:122)
        at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:501)
        at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:170)
        at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:98)
        at org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:950)
        at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:116)
        at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:408)
        at org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1040)
        at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:607)
        at org.apache.tomcat.util.net.JIoEndpoint$SocketProcessor.run(JIoEndpoint.java:313)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
        at java.lang.Thread.run(Thread.java:745)
Caused by: java.io.IOException: Couldn't instantiate
org.apache.zookeeper.ClientCnxnSocketNIO
        at org.apache.zookeeper.ZooKeeper.getClientCnxnSocket(ZooKeeper.java:1778)
        at org.apache.zookeeper.ZooKeeper.<init>(ZooKeeper.java:447)
        at org.apache.zookeeper.ZooKeeper.<init>(ZooKeeper.java:380)
        at org.apache.manifoldcf.core.lockmanager.ZooKeeperConnection.createSession(ZooKeeperConnection.java:74)
        ... 34 more
Caused by: java.io.IOException: Too many open files
        at sun.nio.ch.IOUtil.makePipe(Native Method)
        at sun.nio.ch.EPollSelectorImpl.<init>(EPollSelectorImpl.java:65)
        at sun.nio.ch.EPollSelectorProvider.openSelector(EPollSelectorProvider.java:36)
        at java.nio.channels.Selector.open(Selector.java:227)
        at org.apache.zookeeper.ClientCnxnSocketNIO.<init>(ClientCnxnSocketNIO.java:43)
        at sun.reflect.GeneratedConstructorAccessor27.newInstance(Unknown
Source)
        at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
        at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
        at java.lang.Class.newInstance(Class.java:374)
        at org.apache.zookeeper.ZooKeeper.getClientCnxnSocket(ZooKeeper.java:1775)
        ... 37 more




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


> "Too many open files" IO exception from Rest API
> ------------------------------------------------
>
>                 Key: CONNECTORS-1116
>                 URL: https://issues.apache.org/jira/browse/CONNECTORS-1116
>             Project: ManifoldCF
>          Issue Type: Bug
>          Components: API
>    Affects Versions: ManifoldCF 1.7.2
>         Environment: Ubuntu Linux, Tomcat, Zookeeper
>            Reporter: Adrian Conlon
>
> Using the REST API to get a list of all the job definitions.
> Using lsof, I can see that the total number of open files jumps from around 160 to 4500
as the call is made.
> Also, I should note that there are around 4000 jobs under MCF control.
> Exception below:
> ERROR 2014-11-26 12:46:07,042 (http-bio-8080-exec-9) - Zookeeper initialization error:
Couldn't instantiate org.apache.zookeeper.ClientCnxnSocketNIO
> org.apache.manifoldcf.core.interfaces.ManifoldCFException: Zookeeper initialization error:
Couldn't instantiate org.apache.zookeeper.ClientCnxnSocketNIO
>         at org.apache.manifoldcf.core.lockmanager.ZooKeeperConnection.createSession(ZooKeeperConnection.java:82)
>         at org.apache.manifoldcf.core.lockmanager.ZooKeeperConnection.<init>(ZooKeeperConnection.java:66)
>         at org.apache.manifoldcf.core.lockmanager.ZooKeeperConnectionPool.grab(ZooKeeperConnectionPool.java:48)
>         at org.apache.manifoldcf.core.lockmanager.ZooKeeperLockObject.obtainGlobalReadLock(ZooKeeperLockObject.java:190)
>         at org.apache.manifoldcf.core.lockmanager.LockObject.enterReadLock(LockObject.java:310)
>         at org.apache.manifoldcf.core.lockmanager.LockGate.enterReadLock(LockGate.java:261)
>         at org.apache.manifoldcf.core.lockmanager.BaseLockManager.enter(BaseLockManager.java:1522)
>         at org.apache.manifoldcf.core.lockmanager.BaseLockManager.enterLocks(BaseLockManager.java:803)
>         at org.apache.manifoldcf.core.cachemanager.CacheManager.enterCache(CacheManager.java:299)
>         at org.apache.manifoldcf.core.cachemanager.CacheManager.findObjectsAndExecute(CacheManager.java:100)
>         at org.apache.manifoldcf.crawler.jobs.Jobs.loadMultiple(Jobs.java:926)
>         at org.apache.manifoldcf.crawler.jobs.Jobs.getAll(Jobs.java:774)
>         at org.apache.manifoldcf.crawler.jobs.JobManager.getAllJobs(JobManager.java:760)
>         at org.apache.manifoldcf.crawler.system.ManifoldCF.apiReadJobs(ManifoldCF.java:1299)
>         at org.apache.manifoldcf.crawler.system.ManifoldCF.executeReadCommand(ManifoldCF.java:2919)
>         at org.apache.manifoldcf.apiservlet.APIServlet.executeRead(APIServlet.java:232)
>         at org.apache.manifoldcf.apiservlet.APIServlet.doGet(APIServlet.java:78)
>         at javax.servlet.http.HttpServlet.service(HttpServlet.java:620)
>         at javax.servlet.http.HttpServlet.service(HttpServlet.java:727)
>         at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:303)
>         at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
>         at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:220)
>         at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:122)
>         at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:501)
>         at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:170)
>         at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:98)
>         at org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:950)
>         at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:116)
>         at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:408)
>         at org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1040)
>         at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:607)
>         at org.apache.tomcat.util.net.JIoEndpoint$SocketProcessor.run(JIoEndpoint.java:313)
>         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
>         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
>         at java.lang.Thread.run(Thread.java:745)
> Caused by: java.io.IOException: Couldn't instantiate org.apache.zookeeper.ClientCnxnSocketNIO
>         at org.apache.zookeeper.ZooKeeper.getClientCnxnSocket(ZooKeeper.java:1778)
>         at org.apache.zookeeper.ZooKeeper.<init>(ZooKeeper.java:447)
>         at org.apache.zookeeper.ZooKeeper.<init>(ZooKeeper.java:380)
>         at org.apache.manifoldcf.core.lockmanager.ZooKeeperConnection.createSession(ZooKeeperConnection.java:74)
>         ... 34 more
> Caused by: java.io.IOException: Too many open files
>         at sun.nio.ch.IOUtil.makePipe(Native Method)
>         at sun.nio.ch.EPollSelectorImpl.<init>(EPollSelectorImpl.java:65)
>         at sun.nio.ch.EPollSelectorProvider.openSelector(EPollSelectorProvider.java:36)
>         at java.nio.channels.Selector.open(Selector.java:227)
>         at org.apache.zookeeper.ClientCnxnSocketNIO.<init>(ClientCnxnSocketNIO.java:43)
>         at sun.reflect.GeneratedConstructorAccessor27.newInstance(Unknown Source)
>         at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
>         at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
>         at java.lang.Class.newInstance(Class.java:374)
>         at org.apache.zookeeper.ZooKeeper.getClientCnxnSocket(ZooKeeper.java:1775)
>         ... 37 more



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message