manifoldcf-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Karl Wright <daddy...@gmail.com>
Subject Re: Sharepoint
Date Fri, 22 Feb 2013 18:08:31 GMT
Ok, just tried tomcat7 here against ManifoldCF trunk, using the following steps:

(1) Set up multiprocess database and initialized it
(2) Installed JDK 1.7 and Tomcat 7.0.37, as a service
(3) Set up the -D switch in the service configuration, pointing at
dist/multiprocess-example/properties.xml
(4) Deployed mcf-crawler-ui.war using the tomcat7 manager application
(5) Restarted tomcat
(6) Browsed to the /mcf-crawler-ui application, and created a
repository connection, selecting Sharepoint 2010 (4.0) and localhost
port 80
(7) Got a perfectly reasonable connection status (site not found).

So it does not appear that I can reproduce your problem.

Could you try the same steps please on your setup (skipping the
downloading parts of course), and also using a trunk checkout of
ManifoldCF?  If you see the same thing then it is clear that the class
loader is running into problems only very late in the Axis request.
If you see something different you wil

On Fri, Feb 22, 2013 at 12:31 PM, Karl Wright <daddywri@gmail.com> wrote:
> Actually, since the axis jars are in connector-lib, you are certainly
> picking up jars from there.  It sounds like tomcat is preventing
> access from classes in those jars to classes in the webapp jars, which
> is a pretty strange thing for tomcat to do.
>
> Could you try the following combination:
>
> - leave commons-logging in the war, but also put it into connector-lib
> - make sure there are no other modifications (i.e. take
> commons-discovery and commons-lib out of the other places you have put
> it)
>
> Let's see if that works....
>
> I've got Tomcat 6 here, so I guess it's time to download Tomcat 7 and
> figure out what they all broke.
>
> Karl
>
> On Fri, Feb 22, 2013 at 12:17 PM, Mark Lugert <mlugert@yahoo.com> wrote:
>> Yes, I'm using OOTB manifold.  So it's all there, but in Tomcat it's not
>> getting loaded.  They are runtime dependencies so everything works, until
>> something needs them, such as sharepoint connector.
>>
>> commons-discovery requires commons-logging I think.  The first to fail was
>> commons-discovery.  After dropping that into tomcat/lib then commons-logging
>> was required.  I'll have to test if dropping commons-discovery into
>> mcf-crawler-ui fixes the issue.
>>
>> But having commons-discovery in the connectors-lib doesn't seem to help.
>>
>> -mark
>>
>> From: Karl Wright <daddywri@gmail.com>
>> To: Mark Lugert <mlugert@yahoo.com>
>> Cc: "user@manifoldcf.apache.org" <user@manifoldcf.apache.org>
>> Sent: Friday, February 22, 2013 11:36 AM
>> Subject: Re: Sharepoint
>>
>> The list of files in mcf-crawler-ui.war is the following:
>>
>> C:\wip\mcf\trunk\dist\web\war>"c:\Program Files\Java\jdk1.6.0_37"\bin\jar tf
>> mcf
>> -crawler-ui.war | more
>> META-INF/
>> META-INF/MANIFEST.MF
>> WEB-INF/
>> WEB-INF/jsp/
>> WEB-INF/lib/
>> ManifoldCF-logo.png
>> WEB-INF/jsp/c.tld
>> WEB-INF/jsp/fmt.tld
>> WEB-INF/jsp/sql.tld
>> WEB-INF/jsp/x.tld
>> WEB-INF/lib/commons-codec.jar
>> WEB-INF/lib/commons-collections.jar
>> WEB-INF/lib/commons-el.jar
>> WEB-INF/lib/commons-fileupload.jar
>> WEB-INF/lib/commons-io.jar
>> WEB-INF/lib/commons-lang.jar
>> WEB-INF/lib/commons-logging.jar
>> WEB-INF/lib/derby.jar
>> WEB-INF/lib/derbyLocale_cs.jar
>> WEB-INF/lib/derbyLocale_de_DE.jar
>> WEB-INF/lib/derbyLocale_es.jar
>> WEB-INF/lib/derbyLocale_fr.jar
>> WEB-INF/lib/derbyLocale_hu.jar
>> WEB-INF/lib/derbyLocale_it.jar
>> WEB-INF/lib/derbyLocale_ja_JP.jar
>> WEB-INF/lib/derbyLocale_ko_KR.jar
>> WEB-INF/lib/derbyLocale_pl.jar
>> WEB-INF/lib/derbyLocale_pt_BR.jar
>> WEB-INF/lib/derbyLocale_ru.jar
>> WEB-INF/lib/derbyLocale_zh_CN.jar
>> WEB-INF/lib/derbyLocale_zh_TW.jar
>> WEB-INF/lib/derbynet.jar
>> WEB-INF/lib/derb ytools.jar
>> WEB-INF/lib/hsqldb.jar
>> WEB-INF/lib/httpclient.jar
>> WEB-INF/lib/httpcore.jar
>> WEB-INF/lib/json.jar
>> WEB-INF/lib/jstl.jar
>> WEB-INF/lib/log4j.jar
>> WEB-INF/lib/mcf-agents.jar
>> WEB-INF/lib/mcf-core.jar
>> WEB-INF/lib/mcf-crawler-ui.jar
>> WEB-INF/lib/mcf-pull-agent.jar
>> WEB-INF/lib/mcf-ui-core.jar
>> WEB-INF/lib/postgresql.jar
>> WEB-INF/lib/serializer.jar
>> WEB-INF/lib/slf4j-api.jar
>> WEB-INF/lib/slf4j-simple.jar
>> WEB-INF/lib/velocity.jar
>> WEB-INF/lib/xalan.jar
>> WEB-INF/lib/xercesImpl.jar
>> WEB-INF/lib/xml-apis.jar
>> adminDefaults.jsp
>> adminHeaders.jsp
>> banner.jsp
>> checkAdminLogin.jsp
>> documentstatus.jsp
>> editauthority.jsp
>> editconnection.jsp
>> editjob.jsp
>> editoutput.jsp
>> error.jsp
>> execute.jsp
>> index.jsp
>> listauthorities.jsp
>> listconnections.jsp
>> listjobs.jsp
>> listoutputs.jsp
>> maxactivityreport.jsp
>> maxbandwidthreport.jsp
>> navigation.jsp
>> queuestatus.jsp
>> resultreport.jsp
>> s etupAdminProfile.jsp
>> showjobstatus.jsp
>> simplereport.jsp
>> style.css
>> viewauthority.jsp
>> viewconnection.jsp
>> viewjob.jsp
>> viewoutput.jsp
>> WEB-INF/web.xml
>>
>> While commons-discovery is not present, commons-logging certainly is.
>>
>> And, in the connector-lib area, where the dependencies are picked up
>> for the connectors, sure enough:
>>
>> C:\wip\mcf\trunk\dist\connector-lib>dir commons*
>> Volume in drive C has no label.
>> Volume Serial Number is 143C-F39A
>>
>> Directory of C:\wip\mcf\trunk\dist\connector-lib
>>
>> 02/12/2013  03:22 PM            79,796 commons-discovery.jar
>>               1 File(s)        79,796 bytes
>>               0 Dir(s)  23,952,457,728 bytes free
>>
>>
>> So it should all work, unless you've been messing with the contents of
>> connector-lib, or you've changed your properties.xml to not point at
>> it.  This is the line that does it:
>>
>>   <libdir path="../connector-lib"/>
>>
>> Can you check that all of this stuff is present where I'm describing
>> it in your setup?  If it is, there might be some kind of classloader
>> funkiness going on which we really would need to get to the bottom of.
>>
>> Karl
>>
>> On Fri, Feb 22, 2013 at 11:22 AM, Mark Lugert <mlugert@yahoo.com> wrote:
>>> Hi Karl,
>>>
>>> Ok, figured this out.  There are 2 issues which I think are bugs.
>>>
>>> When running in Tomcat, with the 3 mcf wars in tomcat/webapps and then -D
>>> pointing to /data/manifold/properties.xml.  The sharepoint connector fails
>>> because it's missing 2 jar files from the classpath:
>>>
>>> 1. commons-discovery-0.4.jar
>>> 2. commons-logging.jar
>>>
>>> I added those to tomcat/lib and I got beyond the axis issue I had last
>>> night.
>>>
>>> Now when it continues it fails with the message:
>>> The site at http://ec2-50-16-175-94.compute-1.amazonaws.com/ did not exist
>>
>>>
>>> However, that's not the actual issue.  The actual URL it's failing to
>>> load,
>>> as expected is:
>>>
>>> http://ec2-50-16-175-94.compute-1.amazonaws.com/_vti_bin/MCPermissions.asmx
>>>
>>> Sounds like those commons-*.jar files need to be in the mcf-crawler-ui.war
>>> file and the sharepoint connector needs to print out the correct error
>>> message when it fails to load one of the asmx files it's looking for.
>>>
>>> thanks,
>>> mark
>>>
>>>
>>> From: Karl Wright <daddywri@gmail.com>
>>> To: Mark Lugert <mlugert@yahoo.com>
>>> Cc: "user@manifoldcf.apache.org" <user@manifoldcf.apache.org>
>>> Sent: Friday, February 22, 2013 4:45 AM
>>> Subject: Re: Sharepoint
>>>
>>> Please try checking out, building, and running trunk.  I committed an
>>> explicit CONNECTION_TIMEOUT value of one minute.  Please be aware that
>>> the schema on trunk has changed since 1.1, so just let it use Derby
>>> and don't try to use whatever database you are using for your test.
>>>
>>> Thanks,
>>> Karl
>>>
>>> On Fri, Feb 22, 2013 at 2:16 AM, Mark Lugert <mlugert@yahoo.com> wrote:
>>>> Tested with 1.6.  Still doesn't work for me.  The solr, Alfresco, and
>>>> CMIS
>>>> connectors all work fine.
>>>>
>>>> Running 1.1 of manifold.
>>>>
>>>> I'm running using the multiprocessor setup with tomcat 6.
>>>>
>>>> Any ideas how I can further debug this?
>>>>
>>>> thanks,
>>>> mark
>>>>
>>>> From: Mark Lugert <mlugert@yahoo.com>
>>>> To: Karl Wright <daddywri@gmail.com>
>>>> Cc: "user@manifoldcf.apache.org" <user@manifoldcf.apache.org>
>>>> Sent: Friday, February 22, 2013 1:43 AM
>>>> Subject: Re: Sharepoint
>>>>
>>>> Hi Karl,
>>>>
>>>> Ok, so this is where it's hanging:
>>>>
>>>> public
>>>> ListsWS(String siteUrl, String userName, String password,
>>>> EngineConfiguration configuration, HttpClient httpClient)
>>>> throws java.net.MalformedURLException
>>>> {
>>>> super(configuration);
>>>>
>>>> It never finishes the configuration in the super class.  I don't have the
>>>> code for that, so not sure why.
>>>>
>>>> The server manifold runs on is using Java 1.7.  Could there be an issue
>>>> with
>>>> Axis and Java 1.7?  Has anyone tried the Sharepoint connector using Java
>>>> 1.7?
>>>>
>>>> thanks,
>>>> Mark
>>>>
>>>> From: Karl Wright <daddywri@gmail.com>
>>>> To: Mark Lugert <mlugert@yahoo.com>
>>>> Cc: "user@manifoldcf.apache.org" <user@manifoldcf.apache.org>
>>>> Sent: Thursday, February 21, 2013 7:55 PM
>>>> Subject: Re: Sharepoint
>>>>
>>>> This is exactly where I would expect it to be during check.  If it
>>>> hasn't thrown an exception and is just sitting there, there are only
>>>> two real possibilities.  Both involve socket timeouts.  One is the
>>>> connection timeout, the other the socket timeout proper.  Currently we
>>>> don't set the connection timeout, so I bet that is interpreted as
>>>> being "infinite".  The socket timeout we set to 15 minutes - seems
>>>> high  but that seems to be needed for some sharepoint systems out
>>>> there; they are slow.
>>>>
>>>> I'll open a ticket to set the connection timeout to something
>>>> reasonable.  But basically this means your communication with the
>>>> amazon instance is not properly configured; you're either hitting a
>>>> firewall somewhere (Amazon's?) or the instance's IIS is not running on
>>>> port 80.
>>>>
>>>> Karl
>>>>
>>>> On Thu, Feb 21, 2013 at 7:44 PM, Mark Lugert <mlugert@yahoo.com> wrote:
>>>>> Debuggin this, it's hung up on this call:
>>>>>
>>>>> viewconnection.jsp
>>>>> connectionStatus = c.check();
>>>>>
>>>>> This essentially does a:
>>>>>
>>>>> getSession();
>>>>> URL urlServer =
>>>>> new URL( serverUrl );
>>>>> proxy.checkConnection(
>>>>> "/", supportsItemSecurity );
>>>>>
>>>>> There is pretty much no logging in this entire connector.  I'll go in
>>>>> and
>>>>> add some to see which method, and where, it's having issues.
>>>>>
>>>>> Looking at the code I'd guess the proxy.checkConnection.  However, if
I
>>>>> use
>>>>> the URL it "should" be using via a browser it works fine.
>>>>>
>>>>> Wondering if Amazon is doing something that is causing issues?  Has
>>>>> anyone
>>>>> tried using a Sharepoint in Amazon before?
>>>>>
>>>>> thanks,
>>>>> mark
>>>>>
>>>>> From: Karl Wright <daddywri@gmail.com>
>>>>> To: Mark Lugert <mlugert@yahoo.com>
>>>>> Cc: "user@manifoldcf.apache.org" <user@manifoldcf.apache.org>
>>>>> Sent: Thursday, February 21, 2013 6:13 PM
>>>>> Subject: Re: Sharepoint
>>>>>
>>>>> Ok, just confirmed here that SharePoint 2010 selected with the
>>>>> following parameters gives you something back in less than 1 min:
>>>>>
>>>>> Parameters:    serverLocation=
>>>>> serverPort=
>>>>> serverVersion=4.0
>>>>> userName=domain\foo
>>>>> serverProtocol=http
>>>>> serverName=localhost
>>>>> password=********
>>>>> Connection status:    The site at http://localhost/ did not exist
>>
>>>>>
>>>>> Obviously I didn't try connecting to an Amazon instance with a missing
>>>>> MCPermissions.asmx, but that shouldn't be much different.
>>>>>
>>>>> If the empty response persists, we can assume it's throwing some weird
>>>>> Error and try to catch it in the JSP.  If you are interested in doing
>>>>> that let me know and I'll give you precise instructions.
>>>>>
>>>>> Thanks
>>>>> Karl
>>>>>
>>>>>
>>>>> On Thu, Feb 21, 2013 at 6:02 PM, Mark Lugert <mlugert@yahoo.com>
wrote:
>>>>>> I see no exceptions.  Going to install the service tonight or tomorrow
>>>>>> and
>>>>>> retest.
>>>>>>
>>>>>> Never does return which is weird.  I'll keep digging.
>>>>>>
>>>>>> Sent from my iPhone
>>>>>>
>>>>>> On Feb 21, 2013, at 6:00 PM, Karl Wright <daddywri@gmail.com>
wrote:
>>>>>>
>>>>>>> For SharePoint 2010 and up, MCPermissions.asmx is required, because
>>>>>>> many of the services that worked previously in SharePoint 2003
and
>>>>>>> 2008 no longer work properly on SharePoint 2010.  As far as I
know,
>>>>>>> Amazon does not provide anything other than SharePoint 2010 instances,
>>>>>>> so that is probably going to be an issue for you.
>>>>>>>
>>>>>>> But you *should* get back a connection failure message on the
view
>>>>>>> page.  The reason not may be timeout (may take a minute or two),
or it
>>>>>>> could be something more serious.  Do you see any exceptions printed
in
>>>>>>> either the logs or out to standard out?
>>>>>>>
>>>>>>> Karl
>>>>>>>
>>>>>>> On Thu, Feb 21, 2013 at 5:03 PM, Mark Lugert <mlugert@yahoo.com>
>>>>>>> wrote:
>>>>>>>> Another note.  I did not install MCPermissions.asmx as I
do not need
>>>>>>>> permission information from Sharepoint at this time.  All
the docs
>>>>>>>> are
>>>>>>>> basically public.  It's more about what docs are ready for
publish.
>>>>>>>>
>>>>>>>> Would this cause the problem I'm seeing below?  The docs
make it
>>>>>>>> sound
>>>>>>>> like
>>>>>>>> MCPermissions.asmx is only if you want authority info from
the docs.
>>>>>>>>
>>>>>>>> thanks,
>>>>>>>> mark
>>>>>>>>
>>>>>>>>
>>>>>>>> From: Mark Lugert <mlugert@yahoo.com>
>>>>>>>> To: "user@manifoldcf.apache.org" <user@manifoldcf.apache.org>
>>>>>>>> Sent: Thursday, February 21, 2013 5:00 PM
>>>>>>>> Subject: Sharepoint
>>>>>>>>
>>>>>>>> I'm trying to connect to my sharepoint server, which is on
amazon.
>>>>>>>>
>>>>>>>> There are no sites, just a document library.  I can connect
via a
>>>>>>>> browser
>>>>>>>> just fine, but when I set up the manifold repos connector
and click
>>>>>>>> save
>>>>>>>> I
>>>>>>>> just get a blank screen.  Nothing in the logs and the screen
stays
>>>>>>>> blank.
>>>>>>>>
>>>>>>>> It's sharepoint 2010
>>>>>>>> http
>>>>>>>> my amazon ec2-50-16......amzaonaws.com
>>>>>>>> server port: 80
>>>>>>>> Site Path: <blank> since I don't have any sites
>>>>>>>> user name: <host_name>\Administrator  - this is what
it shows me
>>>>>>>> logged
>>>>>>>> in
>>>>>>>> as via the browser
>>>>>>>> Password.
>>>>>>>> No cert.
>>>>>>>>
>>>>>>>> Any idea why there is nothing in the log and why the screen
goes
>>>>>>>> blank?
>>>>>>>> I
>>>>>>>> turned on wire level debugging and got nothing.
>>>>>>>>
>>>>>>>> thanks,
>>>>>>>> mark
>>>>>>>>
>>>>>>>>
>>>>>
>>>>>
>>>>
>>>>
>>>>
>>>>
>>>
>>>
>>
>>

Mime
View raw message