manifoldcf-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Nikita Ahuja <nik...@smartshore.nl>
Subject Re: Problem in fetching the access tokens from Active Directory in Elastic Search index for FileSystem Connector
Date Mon, 05 Feb 2018 14:09:42 GMT
Hi Karl,


This is Sharepoint 2013 version being used. and it uses the Claim Based
Authentication technique only.
 And the repository connection created is for Sharepoint 2013 version only
[image: Inline image 1]



[image: Inline image 2]


[image: Inline image 3]

Is there anything which is being missed to add in the connector??



Thanks and Regards,
Nikita


On Mon, Feb 5, 2018 at 7:33 PM, Karl Wright <daddywri@gmail.com> wrote:

> The "__nosecurity__" values, for Elastic Search as an output, come from
> ManifoldCF.  So you are definitely getting the access tokens from
> ManifoldCF to ElasticSearch.
>
> The configuration for the Security tab also looks correct.
>
> So at the moment I cannot explain why you are not getting security tokens
> out of SharePoint.  About the only possibility left is whether the wrong
> version of Sharepoint is specified.  Can you describe:
> - What version of Sharepoint this is;
> - Whether it is configured to use Claims Based authorization
>
> And please then take a screen shot when you view the repository
> connection.  Thanks!
>
> Karl
>
>
> On Mon, Feb 5, 2018 at 3:21 AM, Nikita Ahuja <nikita@smartshore.nl> wrote:
>
>> Hi Karl,
>>
>> I have done the configuration of the job as per the *End User
>> Documentation*. Still, I am getting "no security" value in the access
>> tokens in the elastic search index (output of my job).
>>
>> [image: Inline image 2]
>>
>>
>> This is the Output connector for Elastic Search. Is there anything needs
>> to added here?
>>
>> [image: Inline image 1]
>>
>>
>>
>> Also, the security is also enabled.
>>
>> [image: Inline image 3]
>>
>>
>> I am not able to understand where is the actual problem?
>>
>>
>> Thanks and Regards,
>> Nikita
>>
>>
>>
>>
>> On Fri, Feb 2, 2018 at 7:53 PM, Karl Wright <daddywri@gmail.com> wrote:
>>
>>> Hi Nikita,
>>>
>>> If you are seeing *no* tokens indexed then I would check your output
>>> connection and your index configuration.  Custom fields need to be added to
>>> Solr for these tokens to be indexed, if you are using Solr.  There are
>>> instructions in the online documentation on what to add and how to
>>> configure them.
>>>
>>> The other possibility is that you have disabled security for you
>>> Sharepoint job in the "Security" tab.
>>>
>>> Karl
>>>
>>>
>>> On Fri, Feb 2, 2018 at 8:48 AM, Nikita Ahuja <nikita@smartshore.nl>
>>> wrote:
>>>
>>>> Hi Karl,
>>>>
>>>>
>>>> Yes, you are right I am able to fetch the files  but the access tokens
>>>> indexed with the files are not included what is required. And I am using
>>>> the ManifoldCF security model not any custom. Everything is fine but only
>>>> problem is with the token.
>>>> And am not sure what is the reason behind that: is it because the
>>>> Sharepoint site which is used is not with *HTTPS* protocol or any
>>>> other reason?
>>>>
>>>>
>>>> Thanks and Regards,
>>>> Nikita
>>>>
>>>> On Fri, Feb 2, 2018 at 7:05 PM, Karl Wright <daddywri@gmail.com> wrote:
>>>>
>>>>> I am not sure I understand -- are you saying that you can fetch the
>>>>> files but the access tokens indexed with the files do not include what
you
>>>>> require? And that the reason you require it done a certain way is because
>>>>> you intend to apply security in a manner of your own design?
>>>>>
>>>>> The access tokens that are indexed with files in SharePoint are what
>>>>> are needed to enforce security using the ManifoldCF Sharepoint authorities,
>>>>> no more and no less.  The authorities map an individual user to the access
>>>>> tokens needed for that user, which include tokens for the groups as well
as
>>>>> for the user name.  The tokens indexed with the files do NOT include
tokens
>>>>> generated by mapping groups to individual users -- the only tokens indexed
>>>>> for a file are for the groups and for any individual users specified
>>>>> explicitly.
>>>>>
>>>>> I strongly suggest you attempt to apply security fully using the
>>>>> ManifoldCF model rather than trying to do something custom.
>>>>>
>>>>> Karl
>>>>>
>>>>>
>>>>> On Fri, Feb 2, 2018 at 8:04 AM, Nikita Ahuja <nikita@smartshore.nl>
>>>>> wrote:
>>>>>
>>>>>> Hi Karl,
>>>>>>
>>>>>> Thanks a lot for your help.It is working fine now.. I am able to
>>>>>> fetch all the documents in the path but there is a requirement to
get the
>>>>>> access tokens for the files. But,it shows the authority group name
not any
>>>>>> details related to the users who all have access for the document
accessed.
>>>>>>
>>>>>> Please guide for that.
>>>>>>
>>>>>>
>>>>>> [image: Inline image 1]
>>>>>>
>>>>>>
>>>>>> Thanks and Regards,
>>>>>> Nikita
>>>>>>
>>>>>> On Wed, Jan 31, 2018 at 5:55 PM, Karl Wright <daddywri@gmail.com>
>>>>>> wrote:
>>>>>>
>>>>>>> Clearly you are unable to fetch document ACLs from SharePoint.
 This
>>>>>>> has nothing to do with whether your Active Directory is properly
>>>>>>> configured, and has more to do with the MCPermissions plugin.
>>>>>>>
>>>>>>> When you view the SharePoint repository connection, does it say
>>>>>>> "Connection working"?  If not, you need the figure out why.
>>>>>>>
>>>>>>> Here are some possibilities:
>>>>>>> (1) The plugin was properly installed but with the wrong
>>>>>>> permissions.  You must be logged in as an Admin to install it
properly.  If
>>>>>>> that's not done it will mostly work but not be able to retrieve
the
>>>>>>> permissions.
>>>>>>> (2) You may have selected the wrong version of Sharepoint in
the
>>>>>>> repository connection.  Make sure you select the right one or,
once again,
>>>>>>> you won't be able to retrieve permissions.
>>>>>>>
>>>>>>> The documentation says this: "Select your SharePoint server version
>>>>>>> from the pulldown. If you do not select the correct server version,
your
>>>>>>> documents may either be indexed with insufficient security protection,
or
>>>>>>> you may not be able to index any documents. Check with your SharePoint
>>>>>>> system administrator if you are not sure what to select."
>>>>>>>
>>>>>>> Please let me know whether this helps.  Thanks!
>>>>>>> Karl
>>>>>>>
>>>>>>>
>>>>>>> On Wed, Jan 31, 2018 at 6:51 AM, Nikita Ahuja <nikita@smartshore.nl>
>>>>>>> wrote:
>>>>>>>
>>>>>>>> Hi Karl,
>>>>>>>>
>>>>>>>> I have checked the log Files but there is issue with the
ACLs but
>>>>>>>> there is Active Directory already connected with Sharepoint
site. And it is
>>>>>>>> not possible to pass the Access tokens in the connector.
Please provide the
>>>>>>>> suggestion for the problem.
>>>>>>>>
>>>>>>>> [image: Inline image 1]
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> Thanks and Regards,
>>>>>>>> Nikita
>>>>>>>>
>>>>>>>> On Mon, Jan 29, 2018 at 6:55 PM, Karl Wright <daddywri@gmail.com>
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>>> I don't see any path rules there.  That will clearly
not work.
>>>>>>>>>
>>>>>>>>> If you want confirmation of this, you can turn on connector
>>>>>>>>> debugging (set properties.xml property org.apache.manifoldcf.connectors
>>>>>>>>> to "DEBUG"), look at the log output.  Every time the
SharePoint connector
>>>>>>>>> makes a decision to not include something, it logs it.
>>>>>>>>>
>>>>>>>>> Karl
>>>>>>>>>
>>>>>>>>> On Mon, Jan 29, 2018 at 8:07 AM, Nikita Ahuja <
>>>>>>>>> nikita@smartshore.nl> wrote:
>>>>>>>>>
>>>>>>>>>> Hi,
>>>>>>>>>>
>>>>>>>>>> I have checked the documentation and the path configuration
as
>>>>>>>>>> well. But still not able to get any output.
>>>>>>>>>>
>>>>>>>>>> [image: Inline image 1]
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> and the metadata path given is:
>>>>>>>>>> [image: Inline image 2]
>>>>>>>>>>
>>>>>>>>>> Thanks and Regards,
>>>>>>>>>> Nikita
>>>>>>>>>>
>>>>>>>>>> On Mon, Jan 29, 2018 at 4:45 PM, Karl Wright <daddywri@gmail.com>
>>>>>>>>>> wrote:
>>>>>>>>>>
>>>>>>>>>>> Hi,
>>>>>>>>>>>
>>>>>>>>>>> I think your path configuration is incorrect.
 See the end-user
>>>>>>>>>>> documentation.  It's a little tricky because
you must specify a matching
>>>>>>>>>>> rule for both files and paths.
>>>>>>>>>>>
>>>>>>>>>>> Karl
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> On Mon, Jan 29, 2018 at 4:42 AM, Nikita Ahuja
<
>>>>>>>>>>> nikita@smartshore.nl> wrote:
>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>    - Hi Karl,
>>>>>>>>>>>>
>>>>>>>>>>>> There is another query related to Sharepoint
2013 Connector. I
>>>>>>>>>>>> am trying to crawl the files of Sharepoint
On Premise site. But I am not
>>>>>>>>>>>> able to get anything in the output either
in ElasticSearch or in FileShare.
>>>>>>>>>>>> I have followed given steps:
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>    - Created an Authority group for the Active
Directory and
>>>>>>>>>>>>    same is used in Sharepoint Repository
Connection.
>>>>>>>>>>>>
>>>>>>>>>>>> [image: Inline image 2]
>>>>>>>>>>>>
>>>>>>>>>>>>    - Installed the Sharepoint 2013 MCPermission
Plugin and is
>>>>>>>>>>>>    accessible on the site.
>>>>>>>>>>>>
>>>>>>>>>>>> [image: Inline image 3]
>>>>>>>>>>>>
>>>>>>>>>>>>    - Still not able to access any document
from the selected
>>>>>>>>>>>>    folder of Sharepoint Site.
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> Please guide me accordingly.
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> Thanks and Regards,
>>>>>>>>>>>> Nikita
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> On Thu, Jan 25, 2018 at 6:46 PM, Nikita Ahuja
<
>>>>>>>>>>>> nikita@smartshore.nl> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>>> Thanks Karl,
>>>>>>>>>>>>>
>>>>>>>>>>>>> It is working fine now and Also I am
able to get the access
>>>>>>>>>>>>> tokens in the elastic search index.
>>>>>>>>>>>>>
>>>>>>>>>>>>> Thanks a lot for your valuable suggestions.
>>>>>>>>>>>>>
>>>>>>>>>>>>> With Regards,
>>>>>>>>>>>>> Nikita
>>>>>>>>>>>>>
>>>>>>>>>>>>> On Thu, Jan 25, 2018 at 4:42 PM, Karl
Wright <
>>>>>>>>>>>>> daddywri@gmail.com> wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>>> The __nosecurity__ token is generated
by the Elastic Search
>>>>>>>>>>>>>> Connector.  It is produced (correctly)
when no acls reach the connector.
>>>>>>>>>>>>>> This is why I asked you about the
repository connection, because that
>>>>>>>>>>>>>> connection is not producing any ACLs.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> If you are crawling files in a file
system, you must use the
>>>>>>>>>>>>>> JCIFS connector if you want ACLs
to be generated.  The FileSystem connector
>>>>>>>>>>>>>> does not generate ACLs.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Karl
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> On Thu, Jan 25, 2018 at 3:56 AM,
Nikita Ahuja <
>>>>>>>>>>>>>> nikita@smartshore.nl> wrote:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Hi Karl,
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> I have tried for ElasticSearch
version 1.3.1 and 5.6.1 but
>>>>>>>>>>>>>>> still getting this " __nosecurity__"
token and for
>>>>>>>>>>>>>>> Repository connection  File Sytem
is used and for Transformation  "Tika
>>>>>>>>>>>>>>> Content Extractor" and Metadata
adjuster Connector is used.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> [image: Inline image 1]
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Is there any other thing which
I need to perform??
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Thanks and Regards,
>>>>>>>>>>>>>>> Nikita
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> On Thu, Jan 25, 2018 at 2:13
PM, Karl Wright <
>>>>>>>>>>>>>>> daddywri@gmail.com> wrote:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Hi Nikita,
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> You are getting the __nosecurity__
token value transmitted
>>>>>>>>>>>>>>>> and stored, which means your
Elastic Search setup is probably reasonable.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Can you give us details about
your pipeline?  What
>>>>>>>>>>>>>>>> repository connector is this?
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Karl
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> On Thu, Jan 25, 2018 at 1:49
AM, Nikita Ahuja <
>>>>>>>>>>>>>>>> nikita@smartshore.nl>
wrote:
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Hello,
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> I am using ManifoldCF
2.8.1 and trying to access the files
>>>>>>>>>>>>>>>>> from FileShare configured
with the Active Directory and get the data in
>>>>>>>>>>>>>>>>> elastic search index.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>  For this steps followed
are:
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>    - An Authority Group
is created under which Authority
>>>>>>>>>>>>>>>>>    Connection and Repository
connection are created.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> [image: Inline image
3]
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>    - And the elastic
search is created as an Output
>>>>>>>>>>>>>>>>>    Connector and a job
is created for the connection in which a transformation
>>>>>>>>>>>>>>>>>    and metadata connector
is also linked in the
>>>>>>>>>>>>>>>>>    - Which is giving
all the output except the access
>>>>>>>>>>>>>>>>>    token values.
>>>>>>>>>>>>>>>>>    - [image: Inline image
1]
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Please guide to take
appropriate steps for it so that I
>>>>>>>>>>>>>>>>> can get the access tokens
in the elastic search index. Hoping for the
>>>>>>>>>>>>>>>>> solution asap.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Thanks and Regards,
>>>>>>>>>>>>>>>>> Nikita
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>
>>>
>>
>

Mime
View raw message