manifoldcf-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From c.a....@gmx.de
Subject Re: "Connection already in use" when crawling Windows Share repositories
Date Tue, 22 Feb 2011 09:40:11 GMT
Hi Karl,

your guess was right. There has been a problem with too many connections in status "TIME_WAIT".
Even if I set max. connections per JVM to 1 for the Repository and Authority Connection I
hat up to 3500 connections waiting.
When investingating this further I found out that an unbind response from the LDAP server
was missing.

Sometimes this error occurs and sometimes it works fine. It's a bit weird, I wasn't able to
figure out where it came from.

Thanks for your hint.

Carina


-------- Original-Nachricht --------
> Datum: Fri, 18 Feb 2011 06:50:36 -0500
> Von: Karl Wright <daddywri@gmail.com>
> An: connectors-user@incubator.apache.org
> Betreff: Re: "Connection already in use" when crawling Windows Share repositories

> I don't think this is directly a ManifoldCF issue.  My guess is that
> you are running out of file handles.  If you are on Windows, you might
> want to track the number of handles you are using via netstat.  If on
> linux, then lsof is the right tool.
> 
> If you decide that this is indeed what the issue is, then on linux you
> need to either rebuild the kernel, or do stuff with ulimit to set the
> number of handles.  On Windows, I don't know how to do it.  Limiting
> the number of handles used by ManifoldCF is done by setting the number
> of connections under the "Throttling" tab of the various connection
> definitions you create, but that's not going to help with Tomcat.
> 
> It may be worth checking if you aren't closing something.  That could
> easily cause this problem, eventually.  Since it is happening at
> crawling time, the Active Directory Authority is not actually
> communicating with the LDAP server yet.  JCIFS probably is, though.
> 
> Let us know what you find.
> 
> Thanks,
> Karl
> 
> 
> On Fri, Feb 18, 2011 at 6:31 AM,  <c.a.r.e@gmx.de> wrote:
> > I'm running MCF rev. 1071535 with Apache Tomcat 7. Solr and MCF are
> using HTTPS. When crawling data from a Windows Share Connection with Active
> Directory Authority after some time of crawling my job aborts with the status
> "Error: Couldn't set up SSL connection to ingestion API: Address already in
> use: connect".
> >
> > In the mcf logfile I get:
> >
> > [2011-02-18 11:24:33,695]ERROR Exception tossed: Couldn't set up SSL
> connection to ingestion API: Address already in use: connect
> > org.apache.manifoldcf.core.interfaces.ManifoldCFException: Couldn't set
> up SSL connection to ingestion API: Address already in use: connect
> >        at
> org.apache.manifoldcf.agents.output.solr.HttpPoster.createSocket(HttpPoster.java:736)
> >        at
> org.apache.manifoldcf.agents.output.solr.HttpPoster$IngestThread.run(HttpPoster.java:1006)
> > Caused by: java.net.BindException: Address already in use: connect
> >        at java.net.PlainSocketImpl.socketConnect(Native Method)
> >        at
> java.net.PlainSocketImpl.doConnect(PlainSocketImpl.java:333)
> >        at
> java.net.PlainSocketImpl.connectToAddress(PlainSocketImpl.java:195)
> >        at java.net.PlainSocketImpl.connect(PlainSocketImpl.java:182)
> >        at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:366)
> >        at java.net.Socket.connect(Socket.java:529)
> >        at
> com.sun.net.ssl.internal.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:550)
> >        at
> com.sun.net.ssl.internal.ssl.SSLSocketImpl.<init>(SSLSocketImpl.java:353)
> >        at
> com.sun.net.ssl.internal.ssl.SSLSocketFactoryImpl.createSocket(SSLSocketFactoryImpl.java:71)
> >        at
> org.apache.manifoldcf.agents.output.solr.HttpPoster.createSocket(HttpPoster.java:728)
> >        ... 1 more
> >
> > I use the same LDAP user
> > - to authenticate at MCF
> > - login credentials for the Solr Connection
> > - credentials for authority connection
> > - and for connecting to our LDAP server itself (in tomcat JNDI Realm
> config).
> >
> > It seems to me that the connection to LDAP might be a problem. I tried
> to configure connection pooling for tomcat. (I'm afraid I'm not so familiar
> with that.)
> > "-Dcom.sun.jndi.ldap.connect.pool=true
> -Dcom.jndi.ldap.connect.pool.protocol=ssl"
> >
> > Did I miss sth in the config? Or how do I have to configure MCF?
> >
> > Thanks in advance.
> >
> > Carina
> > --
> > GMX DSL Doppel-Flat ab 19,99 Euro/mtl.! Jetzt mit
> > gratis Handy-Flat! http://portal.gmx.net/de/go/dsl
> >

-- 
GMX DSL Doppel-Flat ab 19,99 Euro/mtl.! Jetzt mit 
gratis Handy-Flat! http://portal.gmx.net/de/go/dsl

Mime
View raw message