manifoldcf-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Karl Wright <daddy...@gmail.com>
Subject Re: Problem with reading files from Sharepoint 2010 to manifldcf
Date Wed, 31 Oct 2012 19:43:23 GMT
Please view the job and cut/paste your current rules so we can see
them.  3 "documents" sounds like:

- root
- site
- library

e.g. no documents.
Karl


On Wed, Oct 31, 2012 at 3:37 PM, Fridler, Oren <oren.fridler@hp.com> wrote:
> We're making progress, but it's not working yet.
> I have a path on the job that matches 3 documents (the "Shared Documents" library on
Sharepoint)
> On MCF "Jobs Status" page I see the job is "Done" and indeed 3 documents were processed,
but on Solr server no documents were received.
> I've used this Solr output connection successfully before on this MCF server with MySQL
JDBC connector and with FileSystem connector and  managed to get documents indexed on Solr
so I assume the issue is not with Solr or with the Solr output connector
> Please advise
> Thanks again for your support
> Oren.
>
> -----Original Message-----
> From: Karl Wright [mailto:daddywri@gmail.com]
> Sent: יום ד 31 אוקטובר 2012 19:19
> To: Fridler, Oren
> Cc: user@manifoldcf.apache.org
> Subject: Re: Problem with reading files from Sharepoint 2010 to manifldcf
>
> I have good news - it is apparently now working.
>
> Check your path rules.  You need to have a path that matches the document part of the
path, e.g. xxx/yyy/*.  The end user documentation explains how to set one of these up.
>
> Karl
>
> On Wed, Oct 31, 2012 at 1:12 PM, Fridler, Oren <oren.fridler@hp.com> wrote:
>> Sorry, my bad, I attached the wrong file.
>> Attached is manifoldcf log when 127.0.0.1 is used for sharepoint
>> server Oren -----Original Message-----
>> From: Karl Wright [mailto:daddywri@gmail.com]
>> Sent: יום ד 31 אוקטובר 2012 15:25
>> To: Fridler, Oren
>> Cc: user@manifoldcf.apache.org
>> Subject: Re: Problem with reading files from Sharepoint 2010 to
>> manifldcf
>>
>> The logs you attached have no entries that are dated later than 10/30, so I am uncertain
they are the right ones.
>>
>> I still see the same error when MCPermissions.asmx is invoked.
>>
>> Karl
>>
>> On Wed, Oct 31, 2012 at 9:16 AM, Karl Wright <daddywri@gmail.com> wrote:
>>> Please see below...
>>>
>>> On Wed, Oct 31, 2012 at 8:59 AM, Fridler, Oren <oren.fridler@hp.com> wrote:
>>>> Thanks Karl
>>>> I'll be happy to contribute to the debugging wiki once I have some helpful
insights.
>>>>
>>>> I'm following your advice and sharing the info in case someone encounter
the same issues:
>>>>
>>>> (1) ShrePoint version - I've found 2 copies of
>>>> MicrosoftSharePoint.dll (see below), I opened them with .Net
>>>> Reflector, the first dll's version is 14.0.0.0 and the second is
>>>> 14.900.0.0 C:\>dir /s /b Microsoft.SharePoint.dll C:\Program
>>>> Files\Common Files\Microsoft Shared\Web Server
>>>> Extensions\14\ISAPI\Microsoft.SharePoint.dll
>>>> C:\Program Files\Common Files\Microsoft Shared\Web Server
>>>> Extensions\14\UserCode\assemblies\Microsoft.SharePoint.dll
>>>>
>>>> I don't know which dll is used by my SharePoint 2010, so I
>>>> uninstalled SharePoint - both dlls were removed and after
>>>> re-installed they were back again :(
>>>>
>>>> I installed the manifold sharepoint plugin (setup output attached) and it
went ok without errors.
>>>>
>>>
>>> I think I've seen the 14.900.0.0 - it is the Microsoft Office
>>> extensions to SharePoint.  But as long as the 14.0.0.0 one is
>>> available that is probably fine.
>>>
>>>> (2) Meaning of error - I followed your idea that maybe redirects are causing
the problem, since ManifoldCF is running on the same server where SharePoint is I changed
the URL and replaced the server IP with "localhost" or "127.0.0.1"
>>>> Now I don't get the 1010 error with Web Application cannot be found, still
no files are imported and the logs (attached) contain these 2 errors:
>>>>
>>>> org.apache.axis.ConfigurationException: No service named
>>>> PermissionsSoap is available ...
>>>> org.apache.axis.ConfigurationException: No service named
>>>> http://microsoft.com/sharepoint/webpartpages/GetListItems is
>>>> available
>>>>
>>>
>>> These are just warnings.  They seem to be due to some kind of
>>> mismatch between the wsdl and what the services actually look like.
>>> But just ignore these for now.
>>>
>>> I'll have a look at your logs shortly and get back to you with an
>>> idea what they are telling us.
>>>
>>> Karl
>>>
>>>> I'll continue to investigate, if someone have any idea/help it would
>>>> be great Thanks Oren.
>>>>
>>>>
>>>> -----Original Message-----
>>>> From: Karl Wright [mailto:daddywri@gmail.com]
>>>> Sent: יום ד 31 אוקטובר 2012 09:39
>>>> To: Fridler, Oren; user@manifoldcf.apache.org
>>>> Subject: Re: Problem with reading files from Sharepoint 2010 to
>>>> manifldcf
>>>>
>>>> Hi Oren,
>>>>
>>>> I've been thinking further about your issue, and how many recent kinds of
posts we've been getting which basically amount to people trying to get the manifoldcf-sharepoint-2010
plugin working on their particular SharePoint instance, which has no doubt been installed
and (mis?)configured by someone else at some point in the past.  I think we're going to need
a how-to-debug page where we can gather everyone's experiences together, including diagnostic
approaches and advice.
>>>> There is already a page that anyone can edit in the ManifoldCF wiki, which
is a fine starting point:
>>>> https://cwiki.apache.org/confluence/display/CONNECTORS/Debugging+Con
>>>> n ections .  I hope you will be willing to contribute to this
>>>> effort.
>>>>
>>>> In the meantime, let's go back over your questions below and try to eliminate
them one at a time, in a more systematic fashion.
>>>>
>>>> (1) Version of SharePoint.
>>>>
>>>> To rule out any funkiness here, the obvious thing to do is to find the version
of your sharepoint.dll.  The dll should be in one of the standard locations where assembly
dlls are deployed on your server.
>>>> The assembly name is Microsoft.SharePoint.dll - nothing else, not
>>>> MicrosoftOffice, or anything else.  There are a number of tools for
>>>> determining the .NET version of such DLLs; here's a link that might
>>>> help:
>>>> http://stackoverflow.com/questions/227886/how-do-i-determine-the-dep
>>>> e ndencies-of-a-net-application .  The ManifoldCF-SharePoint-2010
>>>> plugin is built against:
>>>>
>>>> <Reference Include="Microsoft.SharePoint, Version=14.0.0.0,
>>>> Culture=neutral, PublicKeyToken=71e9bce111e9429c,
>>>> processorArchitecture=MSIL" />
>>>>
>>>> ... which can be found in the webservice/MCPermissionsService.csproj
>>>> file in the source package for the service.  The
>>>> ManifoldCF-SharePoint-2007 plugin is, obviously, built against a different
version:
>>>>
>>>> <Reference Include="Microsoft.SharePoint, Version=12.0.0.0,
>>>> Culture=neutral, PublicKeyToken=71e9bce111e9429c,
>>>> processorArchitecture=MSIL" />
>>>>
>>>> (2) Meaning of error
>>>>
>>>> Here's the error again:
>>>> {}Error:<ErrorNumber>1010</ErrorNumber><ErrorMessage>The
Web
>>>> application at http://16.59.60.113 could not be found. Verify that
>>>> you have typed the URL correctly. If the URL should be serving
>>>> existing content, the system administrator may need to add a new
>>>> request URL mapping to the intended
>>>> application.</ErrorMessage><ErrorSource>Microsoft.SharePoint</ErrorS
>>>> o
>>>> urce>
>>>>
>>>> The error code 1010 comes from the plugin, specifically from the GetListItems
method:
>>>>
>>>>             catch (Exception ex)
>>>>             {
>>>>                 EventLog.WriteEntry("MCPermissions.asmx", ex.Message);
>>>>                 throw RaiseException(ex.Message, "1010", ex.Source);
>>>>             }
>>>>
>>>> So, we know we are getting into the plugin correctly, but we furthermore
know that something that is happening in there is not working.  The <ErrorSource> tags
include the assembly from which the error is coming:
>>>>
>>>> Microsoft.SharePoint
>>>>
>>>> The error message, as I pointed out before, is pretty useless when SharePoint
is concerned - there are quite a number of "catchall"
>>>> errors which are more likely to mislead you than help you.  So you have to
look at the source code, which is actually rather small and simple.
>>>>
>>>> Looking at the code itself, and what it is doing, the likely place that the
problem comes from is this:
>>>>
>>>>                 using (SPSite site = new SPSite(SPContext.Current.Web.Url))
>>>>                 {
>>>>                     using (SPWeb oWebsiteRoot = site.OpenWeb())
>>>>                     {
>>>> ...
>>>>
>>>> It seems clear that for some reason your SharePoint instance does not have
a valid SPContext.Current.Web.Url which will permit the plugin reaching the actual sharepoint
logic.  I don't know the reason for that; this is happening internal to SharePoint on that
server.
>>>> Possibilities include a URL redirection, I suppose?  My knowledge of .NET,
and what SharePoint is doing under the covers, is not that strong.  But this is the avenue
I'd pursue.  If you do find that there's a redirection taking place to reach your _vti_bin
directory, try using the final target of the redirection instead of the initial URL, and see
if that helps...
>>>>
>>>> Karl
>>>>
>>>>
>>>> On Tue, Oct 30, 2012 at 11:44 AM, Karl Wright <daddywri@gmail.com>
wrote:
>>>>> Seeing the existence of the service in the browser does not mean it
>>>>> will work.  It only means that the wsdl is coming back from the
>>>>> service.
>>>>>
>>>>>> What can be the reason for this?
>>>>>
>>>>> Unfortunately that is very difficult to determine.  SharePoint
>>>>> tends to return "catchall" errors which are not very meaningful.
>>>>> The server-side event logs may be helpful in figuring out what is
>>>>> going wrong.
>>>>>
>>>>>> Can there be a mismatch between the sharepoint driver on MCF and
the sharepoint server?
>>>>>
>>>>> This is possible if (for instance) you deployed a SharePoint 2010
>>>>> plugin on a SharePoint 2007 server, but if you had a version of
>>>>> SharePoint which was incompatible with the plugin you deployed, I
>>>>> would expect you would have seen errors reported during the plugin
>>>>> installation.  The plugins are built against specific SharePoint
>>>>> dlls with specific version numbers, and .NET enforces a match.  The
>>>>> .bat deployment files though are not very good at telling you that
>>>>> stuff is broken; they don't actually catch the reported errors and
>>>>> stop, so it is possible you may have missed such errors.
>>>>>
>>>>> If there were no errors, I would guess that the problem is probably
>>>>> permissions related.  That is, the plugin may not have permissions
>>>>> to do what it needs to do.  The permissions are granted (as I
>>>>> understand
>>>>> it) based on the user that installs the plugin, so that may be what
>>>>> the issue is.
>>>>>
>>>>> Karl
>>>>>
>>>>>
>>>>> On Tue, Oct 30, 2012 at 11:19 AM, Fridler, Oren <oren.fridler@hp.com>
wrote:
>>>>>> Discovery is not working indeed (sorry I was not clear on this),
I just saw on the sharepoint repository connector UI the status "connection working"
>>>>>>
>>>>>> So if I understand you correctly the soap call to  com.microsoft.sharepoint.webpartpages.PermissionsSoapStub.getListItems(PermissionsSoapStub.java:234)
 is failing? Although I can see the GetListItems operation supported in the browser.
>>>>>> What can be the reason for this?
>>>>>> Can there be a mismatch between the sharepoint driver on MCF and
the sharepoint server?
>>>>>> How do you suggest I continue to investigate?
>>>>>> Thanks
>>>>>> Oren.
>>>>>>
>>>>>>
>>>>>> -----Original Message-----
>>>>>> From: Karl Wright [mailto:daddywri@gmail.com]
>>>>>> Sent: יום ג 30 אוקטובר 2012 17:05
>>>>>> To: Fridler, Oren
>>>>>> Subject: Re: Problem with reading files from Sharepoint 2010 to
>>>>>> manifldcf
>>>>>>
>>>>>> I responded to user@manifoldcf.a.o.  The log disagrees with the idea
that discovery is working.  It seems like the getListItems() part of the service is failing,
and on the very first call too.
>>>>>>
>>>>>> Karl
>>>>>>
>>>>>> On Tue, Oct 30, 2012 at 10:39 AM, Fridler, Oren <oren.fridler@hp.com>
wrote:
>>>>>>> I selected SharePoint 2010.
>>>>>>> There is only one user I used for the SharePoint Server install
and this user is used on MCF SharePoint connection.
>>>>>>> Is there a way to disable permission checking altogether in the
connector and just ask for all documents with the user credentials I entered on the sharepoint
connection? I tried to select secutiry=disabled on the job details but it didn't help.
>>>>>>>
>>>>>>>
>>>>>>> -----Original Message-----
>>>>>>> From: Karl Wright [mailto:daddywri@gmail.com]
>>>>>>> Sent: יום ג 30 אוקטובר 2012 16:26
>>>>>>> To: Fridler, Oren
>>>>>>> Cc: user@manifoldcf.apache.org
>>>>>>> Subject: Re: Problem with reading files from Sharepoint 2010
to
>>>>>>> manifldcf
>>>>>>>
>>>>>>> Hi Oren,
>>>>>>>
>>>>>>> Here's my reasoning:
>>>>>>>
>>>>>>> (1) You would not get "connection working" if you could not access
the MCPermissions service, unless you selected SharePoint 2003, which would then conflict
with other data.
>>>>>>>
>>>>>>> (2) You said that it discovered documents.  That means that the
GetListItems part of the service is working.
>>>>>>>
>>>>>>> (3) You said that you couldn't index any documents, and got an
AXIS exception which terminated the job.  That means you could not retrieve document permissions
(which is what the GetPermissionCollection part of the service does).
>>>>>>>
>>>>>>> (4) The GetPermissionCollection operation uses only one other
service, and it is Permissions.asmx.  So it figured that the problem was likely in reaching
that service, since the complaint was that it couldn't find a service.
>>>>>>>
>>>>>>> Until 10 min ago I did not have internet service back, but I
will confirm this picture in your logs shortly.
>>>>>>>
>>>>>>> The Permissions.asmx service you identify is the correct one;
the question seems to be why the MCPermissions service can't talk to it.
>>>>>>> Could be a permission problem I suppose - perhaps the user you
were logged in as when you installed the service had insufficient permissions or some such?
 Just guessing here...
>>>>>>>
>>>>>>> Karl
>>>>>>>
>>>>>>>
>>>>>>> On Tue, Oct 30, 2012 at 9:19 AM, Fridler, Oren <oren.fridler@hp.com>
wrote:
>>>>>>>> Hi Karl
>>>>>>>> Thank you for your prompt reply,
>>>>>>>>
>>>>>>>> By "SharePoint permissions service" do you refer to this?
  http://16.59.60.113/_vti_bin/Permissions.asmx
>>>>>>>> I was able to open this service, getting the following operations:
>>>>>>>> AddPermission
>>>>>>>> AddPermissionCollection
>>>>>>>> GetPermissionCollection
>>>>>>>> RemovePermission
>>>>>>>> RemovePermissionCollection
>>>>>>>> UpdatePermission
>>>>>>>>
>>>>>>>> BTW, how can you tell from the logs the mcpermissions server
is having trouble reaching SharePoint permissions service?
>>>>>>>> Thanks in advance
>>>>>>>> Oren.
>>>>>>>>
>>>>>>>> -----Original Message-----
>>>>>>>> From: Karl Wright [mailto:daddywri@gmail.com]
>>>>>>>> Sent: יום ג 30 אוקטובר 2012 14:56
>>>>>>>> To: Fridler, Oren; user@manifoldcf.apache.org
>>>>>>>> Subject: RE: Problem with reading files from Sharepoint 2010
to
>>>>>>>> manifldcf
>>>>>>>>
>>>>>>>>
>>>>>>>> Hi Oren,
>>>>>>>>
>>>>>>>> It looks like manifold is able to reach the mcpermissions
service, but the mcpermissions service is having trouble reaching the SharePoint permissions
service, which it needs. Can you access that service, or has it been disabled?
>>>>>>>>
>>>>>>>> Thanks,
>>>>>>>> Karl
>>>>>>>>
>>>>>>>> Sent from my Windows Phone
>>>>>>>> From: Fridler, Oren
>>>>>>>> Sent: 10/30/2012 8:41 AM
>>>>>>>> To: user@manifoldcf.apache.org
>>>>>>>> Subject: Problem with reading files from Sharepoint 2010
to
>>>>>>>> manifldcf
>>>>>>>> 1.0.1
>>>>>>>> Hi
>>>>>>>> I'm using apache-manifoldcf-1.0.1-bin I installed
>>>>>>>> apache-manifoldcf-sharepoint-2010-plugin-0.1  on top of
>>>>>>>> Sharepoint
>>>>>>>> 2010
>>>>>>>>
>>>>>>>> On mcf I managed to create a Sharepoint repository connection
and saw the status is "Connection Working"
>>>>>>>> Also when I create the "Sharepoint to Solr" Job I can see
some of the wiki libraries that I created on SP are available for selection so I assume MCF
is getting this data from SP.
>>>>>>>> But when I start the job it is getting stuck in status "running"
>>>>>>>> forever, the mcf UI shows documents are discovered, some
are processed and some are active, but on Solr side no document is received.
>>>>>>>> On mcf logs I see the error at the end of this email.
>>>>>>>> On my browser I can open http://16.59.60.113 - getting to
SP site, and also http://16.59.60.113/_vti_bin/MCPermissions.asmx  - getting to a page that
lists these 2 services - GetListItems and GetPermissionCollection Attached are the mcf logs
with DEBUG level.
>>>>>>>> Any help or idea what can I do would be highly appreciated.
>>>>>>>> Thanks
>>>>>>>> Oren.
>>>>>>>>
>>>>>>>>
>>>>>>>> AxisFault
>>>>>>>> faultCode: {http://schemas.xmlsoap.org/soap/envelope/}Client
>>>>>>>> faultSubcode:
>>>>>>>>  faultString: The Web application at http://16.59.60.113
could not be found. Verify that you have typed the URL correctly. If the URL should be serving
existing content, the system administrator may need to add a new request URL mapping to the
intended application.
>>>>>>>> faultActor: http://16.59.60.113/_vti_bin/MCPermissions.asmx
>>>>>>>> faultNode:
>>>>>>>>  faultDetail:
>>>>>>>>
>>>>>>>> {}Error:<ErrorNumber>1010</ErrorNumber><ErrorMessage>The
>>>>>>>> Web application at http://16.59.60.113 could not be found.
>>>>>>>> Verify that you have typed the URL correctly. If the URL
should
>>>>>>>> be serving existing content, the system administrator may
need
>>>>>>>> to add a new request URL mapping to the intended
>>>>>>>> application.</ErrorMessage><ErrorSource>Microsoft.SharePoint</Er
>>>>>>>> r
>>>>>>>> or
>>>>>>>> So
>>>>>>>> u
>>>>>>>> rce>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>

Mime
View raw message