manifoldcf-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Karl Wright <daddy...@gmail.com>
Subject Re: Sharepoint
Date Fri, 22 Feb 2013 18:06:16 GMT
Ok, just tried tomcat7 here against ManifoldCF trunk, using the following steps:

(0) Checked out trunk ManifoldCF and built it
(1) Set up multiprocess database and initialized it
(2) Installed JDK 1.7 and Tomcat 7.0.37, as a service
(3) Set up the -D switch in the service configuration, pointing at
dist/multiprocess-example/properties.xml
(4) Deployed mcf-crawler-ui.war using the tomcat7 manager application
(5) Restarted tomcat
(6) Browsed to the /mcf-crawler-ui application, and created a
repository connection, selecting Sharepoint 2010 (4.0) and localhost
port 80
(7) Got a perfectly reasonable connection status (site not found).

So it does not appear that I can readily reproduce your problem with a
straightforward attempt.  Perhaps the error is caused after the
back-and-forth, perhaps there's something else different.

Could you try the same steps please on your setup (skipping the
downloading parts of course), and also using a trunk checkout of
ManifoldCF?  If you see the same thing I do, then please try creating
a connection to your Amazon instance, and see if that fails.

Thanks,
Karl

On Fri, Feb 22, 2013 at 12:31 PM, Karl Wright <daddywri@gmail.com> wrote:
> Actually, since the axis jars are in connector-lib, you are certainly
> picking up jars from there.  It sounds like tomcat is preventing
> access from classes in those jars to classes in the webapp jars, which
> is a pretty strange thing for tomcat to do.
>
> Could you try the following combination:
>
> - leave commons-logging in the war, but also put it into connector-lib
> - make sure there are no other modifications (i.e. take
> commons-discovery and commons-lib out of the other places you have put
> it)
>
> Let's see if that works....
>
> I've got Tomcat 6 here, so I guess it's time to download Tomcat 7 and
> figure out what they all broke.
>
> Karl
>
> On Fri, Feb 22, 2013 at 12:17 PM, Mark Lugert <mlugert@yahoo.com> wrote:
>> Yes, I'm using OOTB manifold.  So it's all there, but in Tomcat it's not
>> getting loaded.  They are runtime dependencies so everything works, until
>> something needs them, such as sharepoint connector.
>>
>> commons-discovery requires commons-logging I think.  The first to fail was
>> commons-discovery.  After dropping that into tomcat/lib then commons-logging
>> was required.  I'll have to test if dropping commons-discovery into
>> mcf-crawler-ui fixes the issue.
>>
>> But having commons-discovery in the connectors-lib doesn't seem to help.
>>
>> -mark
>>
>> From: Karl Wright <daddywri@gmail.com>
>> To: Mark Lugert <mlugert@yahoo.com>
>> Cc: "user@manifoldcf.apache.org" <user@manifoldcf.apache.org>
>> Sent: Friday, February 22, 2013 11:36 AM
>> Subject: Re: Sharepoint
>>
>> The list of files in mcf-crawler-ui.war is the following:
>>
>> C:\wip\mcf\trunk\dist\web\war>"c:\Program Files\Java\jdk1.6.0_37"\bin\jar tf
>> mcf
>> -crawler-ui.war | more
>> META-INF/
>> META-INF/MANIFEST.MF
>> WEB-INF/
>> WEB-INF/jsp/
>> WEB-INF/lib/
>> ManifoldCF-logo.png
>> WEB-INF/jsp/c.tld
>> WEB-INF/jsp/fmt.tld
>> WEB-INF/jsp/sql.tld
>> WEB-INF/jsp/x.tld
>> WEB-INF/lib/commons-codec.jar
>> WEB-INF/lib/commons-collections.jar
>> WEB-INF/lib/commons-el.jar
>> WEB-INF/lib/commons-fileupload.jar
>> WEB-INF/lib/commons-io.jar
>> WEB-INF/lib/commons-lang.jar
>> WEB-INF/lib/commons-logging.jar
>> WEB-INF/lib/derby.jar
>> WEB-INF/lib/derbyLocale_cs.jar
>> WEB-INF/lib/derbyLocale_de_DE.jar
>> WEB-INF/lib/derbyLocale_es.jar
>> WEB-INF/lib/derbyLocale_fr.jar
>> WEB-INF/lib/derbyLocale_hu.jar
>> WEB-INF/lib/derbyLocale_it.jar
>> WEB-INF/lib/derbyLocale_ja_JP.jar
>> WEB-INF/lib/derbyLocale_ko_KR.jar
>> WEB-INF/lib/derbyLocale_pl.jar
>> WEB-INF/lib/derbyLocale_pt_BR.jar
>> WEB-INF/lib/derbyLocale_ru.jar
>> WEB-INF/lib/derbyLocale_zh_CN.jar
>> WEB-INF/lib/derbyLocale_zh_TW.jar
>> WEB-INF/lib/derbynet.jar
>> WEB-INF/lib/derb ytools.jar
>> WEB-INF/lib/hsqldb.jar
>> WEB-INF/lib/httpclient.jar
>> WEB-INF/lib/httpcore.jar
>> WEB-INF/lib/json.jar
>> WEB-INF/lib/jstl.jar
>> WEB-INF/lib/log4j.jar
>> WEB-INF/lib/mcf-agents.jar
>> WEB-INF/lib/mcf-core.jar
>> WEB-INF/lib/mcf-crawler-ui.jar
>> WEB-INF/lib/mcf-pull-agent.jar
>> WEB-INF/lib/mcf-ui-core.jar
>> WEB-INF/lib/postgresql.jar
>> WEB-INF/lib/serializer.jar
>> WEB-INF/lib/slf4j-api.jar
>> WEB-INF/lib/slf4j-simple.jar
>> WEB-INF/lib/velocity.jar
>> WEB-INF/lib/xalan.jar
>> WEB-INF/lib/xercesImpl.jar
>> WEB-INF/lib/xml-apis.jar
>> adminDefaults.jsp
>> adminHeaders.jsp
>> banner.jsp
>> checkAdminLogin.jsp
>> documentstatus.jsp
>> editauthority.jsp
>> editconnection.jsp
>> editjob.jsp
>> editoutput.jsp
>> error.jsp
>> execute.jsp
>> index.jsp
>> listauthorities.jsp
>> listconnections.jsp
>> listjobs.jsp
>> listoutputs.jsp
>> maxactivityreport.jsp
>> maxbandwidthreport.jsp
>> navigation.jsp
>> queuestatus.jsp
>> resultreport.jsp
>> s etupAdminProfile.jsp
>> showjobstatus.jsp
>> simplereport.jsp
>> style.css
>> viewauthority.jsp
>> viewconnection.jsp
>> viewjob.jsp
>> viewoutput.jsp
>> WEB-INF/web.xml
>>
>> While commons-discovery is not present, commons-logging certainly is.
>>
>> And, in the connector-lib area, where the dependencies are picked up
>> for the connectors, sure enough:
>>
>> C:\wip\mcf\trunk\dist\connector-lib>dir commons*
>> Volume in drive C has no label.
>> Volume Serial Number is 143C-F39A
>>
>> Directory of C:\wip\mcf\trunk\dist\connector-lib
>>
>> 02/12/2013  03:22 PM            79,796 commons-discovery.jar
>>               1 File(s)        79,796 bytes
>>               0 Dir(s)  23,952,457,728 bytes free
>>
>>
>> So it should all work, unless you've been messing with the contents of
>> connector-lib, or you've changed your properties.xml to not point at
>> it.  This is the line that does it:
>>
>>   <libdir path="../connector-lib"/>
>>
>> Can you check that all of this stuff is present where I'm describing
>> it in your setup?  If it is, there might be some kind of classloader
>> funkiness going on which we really would need to get to the bottom of.
>>
>> Karl
>>
>> On Fri, Feb 22, 2013 at 11:22 AM, Mark Lugert <mlugert@yahoo.com> wrote:
>>> Hi Karl,
>>>
>>> Ok, figured this out.  There are 2 issues which I think are bugs.
>>>
>>> When running in Tomcat, with the 3 mcf wars in tomcat/webapps and then -D
>>> pointing to /data/manifold/properties.xml.  The sharepoint connector fails
>>> because it's missing 2 jar files from the classpath:
>>>
>>> 1. commons-discovery-0.4.jar
>>> 2. commons-logging.jar
>>>
>>> I added those to tomcat/lib and I got beyond the axis issue I had last
>>> night.
>>>
>>> Now when it continues it fails with the message:
>>> The site at http://ec2-50-16-175-94.compute-1.amazonaws.com/ did not exist
>>
>>>
>>> However, that's not the actual issue.  The actual URL it's failing to
>>> load,
>>> as expected is:
>>>
>>> http://ec2-50-16-175-94.compute-1.amazonaws.com/_vti_bin/MCPermissions.asmx
>>>
>>> Sounds like those commons-*.jar files need to be in the mcf-crawler-ui.war
>>> file and the sharepoint connector needs to print out the correct error
>>> message when it fails to load one of the asmx files it's looking for.
>>>
>>> thanks,
>>> mark
>>>
>>>
>>> From: Karl Wright <daddywri@gmail.com>
>>> To: Mark Lugert <mlugert@yahoo.com>
>>> Cc: "user@manifoldcf.apache.org" <user@manifoldcf.apache.org>
>>> Sent: Friday, February 22, 2013 4:45 AM
>>> Subject: Re: Sharepoint
>>>
>>> Please try checking out, building, and running trunk.  I committed an
>>> explicit CONNECTION_TIMEOUT value of one minute.  Please be aware that
>>> the schema on trunk has changed since 1.1, so just let it use Derby
>>> and don't try to use whatever database you are using for your test.
>>>
>>> Thanks,
>>> Karl
>>>
>>> On Fri, Feb 22, 2013 at 2:16 AM, Mark Lugert <mlugert@yahoo.com> wrote:
>>>> Tested with 1.6.  Still doesn't work for me.  The solr, Alfresco, and
>>>> CMIS
>>>> connectors all work fine.
>>>>
>>>> Running 1.1 of manifold.
>>>>
>>>> I'm running using the multiprocessor setup with tomcat 6.
>>>>
>>>> Any ideas how I can further debug this?
>>>>
>>>> thanks,
>>>> mark
>>>>
>>>> From: Mark Lugert <mlugert@yahoo.com>
>>>> To: Karl Wright <daddywri@gmail.com>
>>>> Cc: "user@manifoldcf.apache.org" <user@manifoldcf.apache.org>
>>>> Sent: Friday, February 22, 2013 1:43 AM
>>>> Subject: Re: Sharepoint
>>>>
>>>> Hi Karl,
>>>>
>>>> Ok, so this is where it's hanging:
>>>>
>>>> public
>>>> ListsWS(String siteUrl, String userName, String password,
>>>> EngineConfiguration configuration, HttpClient httpClient)
>>>> throws java.net.MalformedURLException
>>>> {
>>>> super(configuration);
>>>>
>>>> It never finishes the configuration in the super class.  I don't have the
>>>> code for that, so not sure why.
>>>>
>>>> The server manifold runs on is using Java 1.7.  Could there be an issue
>>>> with
>>>> Axis and Java 1.7?  Has anyone tried the Sharepoint connector using Java
>>>> 1.7?
>>>>
>>>> thanks,
>>>> Mark
>>>>
>>>> From: Karl Wright <daddywri@gmail.com>
>>>> To: Mark Lugert <mlugert@yahoo.com>
>>>> Cc: "user@manifoldcf.apache.org" <user@manifoldcf.apache.org>
>>>> Sent: Thursday, February 21, 2013 7:55 PM
>>>> Subject: Re: Sharepoint
>>>>
>>>> This is exactly where I would expect it to be during check.  If it
>>>> hasn't thrown an exception and is just sitting there, there are only
>>>> two real possibilities.  Both involve socket timeouts.  One is the
>>>> connection timeout, the other the socket timeout proper.  Currently we
>>>> don't set the connection timeout, so I bet that is interpreted as
>>>> being "infinite".  The socket timeout we set to 15 minutes - seems
>>>> high  but that seems to be needed for some sharepoint systems out
>>>> there; they are slow.
>>>>
>>>> I'll open a ticket to set the connection timeout to something
>>>> reasonable.  But basically this means your communication with the
>>>> amazon instance is not properly configured; you're either hitting a
>>>> firewall somewhere (Amazon's?) or the instance's IIS is not running on
>>>> port 80.
>>>>
>>>> Karl
>>>>
>>>> On Thu, Feb 21, 2013 at 7:44 PM, Mark Lugert <mlugert@yahoo.com> wrote:
>>>>> Debuggin this, it's hung up on this call:
>>>>>
>>>>> viewconnection.jsp
>>>>> connectionStatus = c.check();
>>>>>
>>>>> This essentially does a:
>>>>>
>>>>> getSession();
>>>>> URL urlServer =
>>>>> new URL( serverUrl );
>>>>> proxy.checkConnection(
>>>>> "/", supportsItemSecurity );
>>>>>
>>>>> There is pretty much no logging in this entire connector.  I'll go in
>>>>> and
>>>>> add some to see which method, and where, it's having issues.
>>>>>
>>>>> Looking at the code I'd guess the proxy.checkConnection.  However, if
I
>>>>> use
>>>>> the URL it "should" be using via a browser it works fine.
>>>>>
>>>>> Wondering if Amazon is doing something that is causing issues?  Has
>>>>> anyone
>>>>> tried using a Sharepoint in Amazon before?
>>>>>
>>>>> thanks,
>>>>> mark
>>>>>
>>>>> From: Karl Wright <daddywri@gmail.com>
>>>>> To: Mark Lugert <mlugert@yahoo.com>
>>>>> Cc: "user@manifoldcf.apache.org" <user@manifoldcf.apache.org>
>>>>> Sent: Thursday, February 21, 2013 6:13 PM
>>>>> Subject: Re: Sharepoint
>>>>>
>>>>> Ok, just confirmed here that SharePoint 2010 selected with the
>>>>> following parameters gives you something back in less than 1 min:
>>>>>
>>>>> Parameters:    serverLocation=
>>>>> serverPort=
>>>>> serverVersion=4.0
>>>>> userName=domain\foo
>>>>> serverProtocol=http
>>>>> serverName=localhost
>>>>> password=********
>>>>> Connection status:    The site at http://localhost/ did not exist
>>
>>>>>
>>>>> Obviously I didn't try connecting to an Amazon instance with a missing
>>>>> MCPermissions.asmx, but that shouldn't be much different.
>>>>>
>>>>> If the empty response persists, we can assume it's throwing some weird
>>>>> Error and try to catch it in the JSP.  If you are interested in doing
>>>>> that let me know and I'll give you precise instructions.
>>>>>
>>>>> Thanks
>>>>> Karl
>>>>>
>>>>>
>>>>> On Thu, Feb 21, 2013 at 6:02 PM, Mark Lugert <mlugert@yahoo.com>
wrote:
>>>>>> I see no exceptions.  Going to install the service tonight or tomorrow
>>>>>> and
>>>>>> retest.
>>>>>>
>>>>>> Never does return which is weird.  I'll keep digging.
>>>>>>
>>>>>> Sent from my iPhone
>>>>>>
>>>>>> On Feb 21, 2013, at 6:00 PM, Karl Wright <daddywri@gmail.com>
wrote:
>>>>>>
>>>>>>> For SharePoint 2010 and up, MCPermissions.asmx is required, because
>>>>>>> many of the services that worked previously in SharePoint 2003
and
>>>>>>> 2008 no longer work properly on SharePoint 2010.  As far as I
know,
>>>>>>> Amazon does not provide anything other than SharePoint 2010 instances,
>>>>>>> so that is probably going to be an issue for you.
>>>>>>>
>>>>>>> But you *should* get back a connection failure message on the
view
>>>>>>> page.  The reason not may be timeout (may take a minute or two),
or it
>>>>>>> could be something more serious.  Do you see any exceptions printed
in
>>>>>>> either the logs or out to standard out?
>>>>>>>
>>>>>>> Karl
>>>>>>>
>>>>>>> On Thu, Feb 21, 2013 at 5:03 PM, Mark Lugert <mlugert@yahoo.com>
>>>>>>> wrote:
>>>>>>>> Another note.  I did not install MCPermissions.asmx as I
do not need
>>>>>>>> permission information from Sharepoint at this time.  All
the docs
>>>>>>>> are
>>>>>>>> basically public.  It's more about what docs are ready for
publish.
>>>>>>>>
>>>>>>>> Would this cause the problem I'm seeing below?  The docs
make it
>>>>>>>> sound
>>>>>>>> like
>>>>>>>> MCPermissions.asmx is only if you want authority info from
the docs.
>>>>>>>>
>>>>>>>> thanks,
>>>>>>>> mark
>>>>>>>>
>>>>>>>>
>>>>>>>> From: Mark Lugert <mlugert@yahoo.com>
>>>>>>>> To: "user@manifoldcf.apache.org" <user@manifoldcf.apache.org>
>>>>>>>> Sent: Thursday, February 21, 2013 5:00 PM
>>>>>>>> Subject: Sharepoint
>>>>>>>>
>>>>>>>> I'm trying to connect to my sharepoint server, which is on
amazon.
>>>>>>>>
>>>>>>>> There are no sites, just a document library.  I can connect
via a
>>>>>>>> browser
>>>>>>>> just fine, but when I set up the manifold repos connector
and click
>>>>>>>> save
>>>>>>>> I
>>>>>>>> just get a blank screen.  Nothing in the logs and the screen
stays
>>>>>>>> blank.
>>>>>>>>
>>>>>>>> It's sharepoint 2010
>>>>>>>> http
>>>>>>>> my amazon ec2-50-16......amzaonaws.com
>>>>>>>> server port: 80
>>>>>>>> Site Path: <blank> since I don't have any sites
>>>>>>>> user name: <host_name>\Administrator  - this is what
it shows me
>>>>>>>> logged
>>>>>>>> in
>>>>>>>> as via the browser
>>>>>>>> Password.
>>>>>>>> No cert.
>>>>>>>>
>>>>>>>> Any idea why there is nothing in the log and why the screen
goes
>>>>>>>> blank?
>>>>>>>> I
>>>>>>>> turned on wire level debugging and got nothing.
>>>>>>>>
>>>>>>>> thanks,
>>>>>>>> mark
>>>>>>>>
>>>>>>>>
>>>>>
>>>>>
>>>>
>>>>
>>>>
>>>>
>>>
>>>
>>
>>

Mime
View raw message