Mailing-List: contact connectors-user-help@incubator.apache.org; run by ezmlm
Precedence: bulk
Reply-To: connectors-user@incubator.apache.org
Received-SPF: pass (nike.apache.org: domain of daddywri@gmail.com designates
 209.85.216.175 as permitted sender)
DomainKey-Signature: a=rsa-sha1; c=nofws;
        d=gmail.com; s=gamma;
        h=mime-version:in-reply-to:references:date:message-id:subject:from:to
         :content-type:content-transfer-encoding;
        b=E+BNZcAMv+YwFZjzhYf52U/vNKURAqesfhxXoXLQVxy4PPZQ1mPvUvXpI9kjVHt3MX
         2FRNr5lAhKkWEvvZCs0RXk+tkgvX1zO7dzH8sQUJc5rXFxLHgA0VFDLNPYZR8ea++am+
         icfs/3FZb6HVk2Aawu7LBKKYqk+L8y8QbCf3U=
MIME-Version: 1.0
In-Reply-To: <20110218113147.118250@gmx.net>
References: <20110218113147.118250@gmx.net>
Date: Fri, 18 Feb 2011 06:50:36 -0500
Message-ID: <AANLkTikcL2rq-sRZTXDEHaZ4xcek0dt_YaNZNc6ZgmNs@mail.gmail.com>
Subject: Re: "Connection already in use" when crawling Windows Share
 repositories
From: Karl Wright <daddywri@gmail.com>
To: connectors-user@incubator.apache.org
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable

I don't think this is directly a ManifoldCF issue.  My guess is that
you are running out of file handles.  If you are on Windows, you might
want to track the number of handles you are using via netstat.  If on
linux, then lsof is the right tool.

If you decide that this is indeed what the issue is, then on linux you
need to either rebuild the kernel, or do stuff with ulimit to set the
number of handles.  On Windows, I don't know how to do it.  Limiting
the number of handles used by ManifoldCF is done by setting the number
of connections under the "Throttling" tab of the various connection
definitions you create, but that's not going to help with Tomcat.

It may be worth checking if you aren't closing something.  That could
easily cause this problem, eventually.  Since it is happening at
crawling time, the Active Directory Authority is not actually
communicating with the LDAP server yet.  JCIFS probably is, though.

Let us know what you find.

Thanks,
Karl


On Fri, Feb 18, 2011 at 6:31 AM,  <c.a.r.e@gmx.de> wrote:
> I'm running MCF rev. 1071535 with Apache Tomcat 7. Solr and MCF are using=
 HTTPS. When crawling data from a Windows Share Connection with Active Dire=
ctory Authority after some time of crawling my job aborts with the status "=
Error: Couldn't set up SSL connection to ingestion API: Address already in =
use: connect".
>
> In the mcf logfile I get:
>
> [2011-02-18 11:24:33,695]ERROR Exception tossed: Couldn't set up SSL conn=
ection to ingestion API: Address already in use: connect
> org.apache.manifoldcf.core.interfaces.ManifoldCFException: Couldn't set u=
p SSL connection to ingestion API: Address already in use: connect
> =A0 =A0 =A0 =A0at org.apache.manifoldcf.agents.output.solr.HttpPoster.cre=
ateSocket(HttpPoster.java:736)
> =A0 =A0 =A0 =A0at org.apache.manifoldcf.agents.output.solr.HttpPoster$Ing=
estThread.run(HttpPoster.java:1006)
> Caused by: java.net.BindException: Address already in use: connect
> =A0 =A0 =A0 =A0at java.net.PlainSocketImpl.socketConnect(Native Method)
> =A0 =A0 =A0 =A0at java.net.PlainSocketImpl.doConnect(PlainSocketImpl.java=
:333)
> =A0 =A0 =A0 =A0at java.net.PlainSocketImpl.connectToAddress(PlainSocketIm=
pl.java:195)
> =A0 =A0 =A0 =A0at java.net.PlainSocketImpl.connect(PlainSocketImpl.java:1=
82)
> =A0 =A0 =A0 =A0at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:3=
66)
> =A0 =A0 =A0 =A0at java.net.Socket.connect(Socket.java:529)
> =A0 =A0 =A0 =A0at com.sun.net.ssl.internal.ssl.SSLSocketImpl.connect(SSLS=
ocketImpl.java:550)
> =A0 =A0 =A0 =A0at com.sun.net.ssl.internal.ssl.SSLSocketImpl.<init>(SSLSo=
cketImpl.java:353)
> =A0 =A0 =A0 =A0at com.sun.net.ssl.internal.ssl.SSLSocketFactoryImpl.creat=
eSocket(SSLSocketFactoryImpl.java:71)
> =A0 =A0 =A0 =A0at org.apache.manifoldcf.agents.output.solr.HttpPoster.cre=
ateSocket(HttpPoster.java:728)
> =A0 =A0 =A0 =A0... 1 more
>
> I use the same LDAP user
> - to authenticate at MCF
> - login credentials for the Solr Connection
> - credentials for authority connection
> - and for connecting to our LDAP server itself (in tomcat JNDI Realm conf=
ig).
>
> It seems to me that the connection to LDAP might be a problem. I tried to=
 configure connection pooling for tomcat. (I'm afraid I'm not so familiar w=
ith that.)
> "-Dcom.sun.jndi.ldap.connect.pool=3Dtrue -Dcom.jndi.ldap.connect.pool.pro=
tocol=3Dssl"
>
> Did I miss sth in the config? Or how do I have to configure MCF?
>
> Thanks in advance.
>
> Carina
> --
> GMX DSL Doppel-Flat ab 19,99 Euro/mtl.! Jetzt mit
> gratis Handy-Flat! http://portal.gmx.net/de/go/dsl
>