manifoldcf-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
Subject [CONF] Lucene Connector Framework > Debugging Connections
Date Wed, 12 May 2010 13:55:00 GMT
Space: Lucene Connector Framework (
Page: Debugging Connections (

Added by Karl Wright:
h1. Debugging Connections

LCF's various repository and authority connections often have to operate in complex enterprise
environments, which have a myriad of different configuration options and pathways.  It can
therefore be extremely challenging to get a connection working.  Many different debugging
techniques may need to come into play to determine the difficulty.  Worse, the kinds of techniques
involved depend strongly on the underlying repository technology.

This document is meant to give ideas to those who get stuck with the job of figuring out why
an LCF connection isn't working as planned.  It cannot obviously be a comprehensive manual,
but it can suggest avenues that might be profitable to pursue.

h3. Debugging JCIFS connection types

JCIFS connections operate in the Windows world, and need to resolve several things in order
to function properly.  These are:

* Machine names
* User names

DFS, which basically is Window's system of cross-machine node referrals, requires that the
connection both be able to resolve the name of the reference (which means that the crawling
server's DNS setup has to be compatible with those for Windows).  It also requires that the
crawling server be able to authenticate reliably against all machines that host DFS content.
 The easiest way to guarantee that this will be the case is to join the crawling server to
the local AD domain.

If you find that a JCIFS connection is not working as planned, you can try the following approaches
to resolving the problem:

* Turn on connector-level debugging.  To do that, add "org.apache.lcf.connectors=DEBUG" to
the properties.ini file, and restart whatever services are appropriate.
* Obtain packet captures, using tools such as tcpdump and/or Wireshark.  This can give a clue
as to what is going wrong.

h3. Debugging SharePoint connection types

The SharePoint connection type works through IIS.  IIS can be configured in many ways that
are not helpful to the SharePoint connector.  For example, redirections that occur when a
SharePoint connection accesses a SharePoint server are bad news, because redirections of POST
requests (which is what SOAP uses) are not well supported in the commons-httpclient library.
 Other examples of configuration problems include tunnels, because IIS may not properly authenticate
connections that are made in this manner.  Finally, sometimes the order in which SharePoint
is installed and Windows security policies are modified can inadvertantly break some specific
SharePoint web services that the connection type relies on.

Subtle problems of these kinds may require saintly patience to work through, because you do
not usually get very good feedback from SharePoint web services as to the underlying reason
for the failure.  The tools found to be helpful include the following:

* Wire-level debugging.  In your logging properties file, add "log4j.logger.httpclient.wire=DEBUG",
or "log4j.logger.httpclient.wire.content=DEBUG", and restart the appropriate service to make
the logging take effect.
* Windows event logs.  Depending on the problem, the security log might be appropriate; for
other problems, the application log would be better.
* Packet captures, using tcpdump and/or Wireshark.  This helps finding the causes for unexpected
redirections, or other http-gated issues.

Change your notification preferences:

View raw message