manifoldcf-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Karl Wright <daddy...@gmail.com>
Subject Re: Problem in fetching the access tokens from Active Directory in Elastic Search index for FileSystem Connector
Date Mon, 05 Feb 2018 14:03:04 GMT
The "__nosecurity__" values, for Elastic Search as an output, come from
ManifoldCF.  So you are definitely getting the access tokens from
ManifoldCF to ElasticSearch.

The configuration for the Security tab also looks correct.

So at the moment I cannot explain why you are not getting security tokens
out of SharePoint.  About the only possibility left is whether the wrong
version of Sharepoint is specified.  Can you describe:
- What version of Sharepoint this is;
- Whether it is configured to use Claims Based authorization

And please then take a screen shot when you view the repository
connection.  Thanks!

Karl


On Mon, Feb 5, 2018 at 3:21 AM, Nikita Ahuja <nikita@smartshore.nl> wrote:

> Hi Karl,
>
> I have done the configuration of the job as per the *End User
> Documentation*. Still, I am getting "no security" value in the access
> tokens in the elastic search index (output of my job).
>
> [image: Inline image 2]
>
>
> This is the Output connector for Elastic Search. Is there anything needs
> to added here?
>
> [image: Inline image 1]
>
>
>
> Also, the security is also enabled.
>
> [image: Inline image 3]
>
>
> I am not able to understand where is the actual problem?
>
>
> Thanks and Regards,
> Nikita
>
>
>
>
> On Fri, Feb 2, 2018 at 7:53 PM, Karl Wright <daddywri@gmail.com> wrote:
>
>> Hi Nikita,
>>
>> If you are seeing *no* tokens indexed then I would check your output
>> connection and your index configuration.  Custom fields need to be added to
>> Solr for these tokens to be indexed, if you are using Solr.  There are
>> instructions in the online documentation on what to add and how to
>> configure them.
>>
>> The other possibility is that you have disabled security for you
>> Sharepoint job in the "Security" tab.
>>
>> Karl
>>
>>
>> On Fri, Feb 2, 2018 at 8:48 AM, Nikita Ahuja <nikita@smartshore.nl>
>> wrote:
>>
>>> Hi Karl,
>>>
>>>
>>> Yes, you are right I am able to fetch the files  but the access tokens
>>> indexed with the files are not included what is required. And I am using
>>> the ManifoldCF security model not any custom. Everything is fine but only
>>> problem is with the token.
>>> And am not sure what is the reason behind that: is it because the
>>> Sharepoint site which is used is not with *HTTPS* protocol or any other
>>> reason?
>>>
>>>
>>> Thanks and Regards,
>>> Nikita
>>>
>>> On Fri, Feb 2, 2018 at 7:05 PM, Karl Wright <daddywri@gmail.com> wrote:
>>>
>>>> I am not sure I understand -- are you saying that you can fetch the
>>>> files but the access tokens indexed with the files do not include what you
>>>> require? And that the reason you require it done a certain way is because
>>>> you intend to apply security in a manner of your own design?
>>>>
>>>> The access tokens that are indexed with files in SharePoint are what
>>>> are needed to enforce security using the ManifoldCF Sharepoint authorities,
>>>> no more and no less.  The authorities map an individual user to the access
>>>> tokens needed for that user, which include tokens for the groups as well
as
>>>> for the user name.  The tokens indexed with the files do NOT include tokens
>>>> generated by mapping groups to individual users -- the only tokens indexed
>>>> for a file are for the groups and for any individual users specified
>>>> explicitly.
>>>>
>>>> I strongly suggest you attempt to apply security fully using the
>>>> ManifoldCF model rather than trying to do something custom.
>>>>
>>>> Karl
>>>>
>>>>
>>>> On Fri, Feb 2, 2018 at 8:04 AM, Nikita Ahuja <nikita@smartshore.nl>
>>>> wrote:
>>>>
>>>>> Hi Karl,
>>>>>
>>>>> Thanks a lot for your help.It is working fine now.. I am able to fetch
>>>>> all the documents in the path but there is a requirement to get the access
>>>>> tokens for the files. But,it shows the authority group name not any details
>>>>> related to the users who all have access for the document accessed.
>>>>>
>>>>> Please guide for that.
>>>>>
>>>>>
>>>>> [image: Inline image 1]
>>>>>
>>>>>
>>>>> Thanks and Regards,
>>>>> Nikita
>>>>>
>>>>> On Wed, Jan 31, 2018 at 5:55 PM, Karl Wright <daddywri@gmail.com>
>>>>> wrote:
>>>>>
>>>>>> Clearly you are unable to fetch document ACLs from SharePoint.  This
>>>>>> has nothing to do with whether your Active Directory is properly
>>>>>> configured, and has more to do with the MCPermissions plugin.
>>>>>>
>>>>>> When you view the SharePoint repository connection, does it say
>>>>>> "Connection working"?  If not, you need the figure out why.
>>>>>>
>>>>>> Here are some possibilities:
>>>>>> (1) The plugin was properly installed but with the wrong
>>>>>> permissions.  You must be logged in as an Admin to install it properly.
 If
>>>>>> that's not done it will mostly work but not be able to retrieve the
>>>>>> permissions.
>>>>>> (2) You may have selected the wrong version of Sharepoint in the
>>>>>> repository connection.  Make sure you select the right one or, once
again,
>>>>>> you won't be able to retrieve permissions.
>>>>>>
>>>>>> The documentation says this: "Select your SharePoint server version
>>>>>> from the pulldown. If you do not select the correct server version,
your
>>>>>> documents may either be indexed with insufficient security protection,
or
>>>>>> you may not be able to index any documents. Check with your SharePoint
>>>>>> system administrator if you are not sure what to select."
>>>>>>
>>>>>> Please let me know whether this helps.  Thanks!
>>>>>> Karl
>>>>>>
>>>>>>
>>>>>> On Wed, Jan 31, 2018 at 6:51 AM, Nikita Ahuja <nikita@smartshore.nl>
>>>>>> wrote:
>>>>>>
>>>>>>> Hi Karl,
>>>>>>>
>>>>>>> I have checked the log Files but there is issue with the ACLs
but
>>>>>>> there is Active Directory already connected with Sharepoint site.
And it is
>>>>>>> not possible to pass the Access tokens in the connector. Please
provide the
>>>>>>> suggestion for the problem.
>>>>>>>
>>>>>>> [image: Inline image 1]
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> Thanks and Regards,
>>>>>>> Nikita
>>>>>>>
>>>>>>> On Mon, Jan 29, 2018 at 6:55 PM, Karl Wright <daddywri@gmail.com>
>>>>>>> wrote:
>>>>>>>
>>>>>>>> I don't see any path rules there.  That will clearly not
work.
>>>>>>>>
>>>>>>>> If you want confirmation of this, you can turn on connector
>>>>>>>> debugging (set properties.xml property org.apache.manifoldcf.connectors
>>>>>>>> to "DEBUG"), look at the log output.  Every time the SharePoint
connector
>>>>>>>> makes a decision to not include something, it logs it.
>>>>>>>>
>>>>>>>> Karl
>>>>>>>>
>>>>>>>> On Mon, Jan 29, 2018 at 8:07 AM, Nikita Ahuja <nikita@smartshore.nl
>>>>>>>> > wrote:
>>>>>>>>
>>>>>>>>> Hi,
>>>>>>>>>
>>>>>>>>> I have checked the documentation and the path configuration
as
>>>>>>>>> well. But still not able to get any output.
>>>>>>>>>
>>>>>>>>> [image: Inline image 1]
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> and the metadata path given is:
>>>>>>>>> [image: Inline image 2]
>>>>>>>>>
>>>>>>>>> Thanks and Regards,
>>>>>>>>> Nikita
>>>>>>>>>
>>>>>>>>> On Mon, Jan 29, 2018 at 4:45 PM, Karl Wright <daddywri@gmail.com>
>>>>>>>>> wrote:
>>>>>>>>>
>>>>>>>>>> Hi,
>>>>>>>>>>
>>>>>>>>>> I think your path configuration is incorrect.  See
the end-user
>>>>>>>>>> documentation.  It's a little tricky because you
must specify a matching
>>>>>>>>>> rule for both files and paths.
>>>>>>>>>>
>>>>>>>>>> Karl
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> On Mon, Jan 29, 2018 at 4:42 AM, Nikita Ahuja <
>>>>>>>>>> nikita@smartshore.nl> wrote:
>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>    - Hi Karl,
>>>>>>>>>>>
>>>>>>>>>>> There is another query related to Sharepoint
2013 Connector. I
>>>>>>>>>>> am trying to crawl the files of Sharepoint On
Premise site. But I am not
>>>>>>>>>>> able to get anything in the output either in
ElasticSearch or in FileShare.
>>>>>>>>>>> I have followed given steps:
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>    - Created an Authority group for the Active
Directory and
>>>>>>>>>>>    same is used in Sharepoint Repository Connection.
>>>>>>>>>>>
>>>>>>>>>>> [image: Inline image 2]
>>>>>>>>>>>
>>>>>>>>>>>    - Installed the Sharepoint 2013 MCPermission
Plugin and is
>>>>>>>>>>>    accessible on the site.
>>>>>>>>>>>
>>>>>>>>>>> [image: Inline image 3]
>>>>>>>>>>>
>>>>>>>>>>>    - Still not able to access any document from
the selected
>>>>>>>>>>>    folder of Sharepoint Site.
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> Please guide me accordingly.
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> Thanks and Regards,
>>>>>>>>>>> Nikita
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> On Thu, Jan 25, 2018 at 6:46 PM, Nikita Ahuja
<
>>>>>>>>>>> nikita@smartshore.nl> wrote:
>>>>>>>>>>>
>>>>>>>>>>>> Thanks Karl,
>>>>>>>>>>>>
>>>>>>>>>>>> It is working fine now and Also I am able
to get the access
>>>>>>>>>>>> tokens in the elastic search index.
>>>>>>>>>>>>
>>>>>>>>>>>> Thanks a lot for your valuable suggestions.
>>>>>>>>>>>>
>>>>>>>>>>>> With Regards,
>>>>>>>>>>>> Nikita
>>>>>>>>>>>>
>>>>>>>>>>>> On Thu, Jan 25, 2018 at 4:42 PM, Karl Wright
<
>>>>>>>>>>>> daddywri@gmail.com> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>>> The __nosecurity__ token is generated
by the Elastic Search
>>>>>>>>>>>>> Connector.  It is produced (correctly)
when no acls reach the connector.
>>>>>>>>>>>>> This is why I asked you about the repository
connection, because that
>>>>>>>>>>>>> connection is not producing any ACLs.
>>>>>>>>>>>>>
>>>>>>>>>>>>> If you are crawling files in a file system,
you must use the
>>>>>>>>>>>>> JCIFS connector if you want ACLs to be
generated.  The FileSystem connector
>>>>>>>>>>>>> does not generate ACLs.
>>>>>>>>>>>>>
>>>>>>>>>>>>> Karl
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> On Thu, Jan 25, 2018 at 3:56 AM, Nikita
Ahuja <
>>>>>>>>>>>>> nikita@smartshore.nl> wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>>> Hi Karl,
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> I have tried for ElasticSearch version
1.3.1 and 5.6.1 but
>>>>>>>>>>>>>> still getting this " __nosecurity__"
token and for
>>>>>>>>>>>>>> Repository connection  File Sytem
is used and for Transformation  "Tika
>>>>>>>>>>>>>> Content Extractor" and Metadata adjuster
Connector is used.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> [image: Inline image 1]
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Is there any other thing which I
need to perform??
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Thanks and Regards,
>>>>>>>>>>>>>> Nikita
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> On Thu, Jan 25, 2018 at 2:13 PM,
Karl Wright <
>>>>>>>>>>>>>> daddywri@gmail.com> wrote:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Hi Nikita,
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> You are getting the __nosecurity__
token value transmitted
>>>>>>>>>>>>>>> and stored, which means your
Elastic Search setup is probably reasonable.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Can you give us details about
your pipeline?  What
>>>>>>>>>>>>>>> repository connector is this?
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Karl
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> On Thu, Jan 25, 2018 at 1:49
AM, Nikita Ahuja <
>>>>>>>>>>>>>>> nikita@smartshore.nl> wrote:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Hello,
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> I am using ManifoldCF 2.8.1
and trying to access the files
>>>>>>>>>>>>>>>> from FileShare configured
with the Active Directory and get the data in
>>>>>>>>>>>>>>>> elastic search index.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>  For this steps followed
are:
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>    - An Authority Group is
created under which Authority
>>>>>>>>>>>>>>>>    Connection and Repository
connection are created.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> [image: Inline image 3]
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>    - And the elastic search
is created as an Output
>>>>>>>>>>>>>>>>    Connector and a job is
created for the connection in which a transformation
>>>>>>>>>>>>>>>>    and metadata connector
is also linked in the
>>>>>>>>>>>>>>>>    - Which is giving all
the output except the access
>>>>>>>>>>>>>>>>    token values.
>>>>>>>>>>>>>>>>    - [image: Inline image
1]
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Please guide to take appropriate
steps for it so that I can
>>>>>>>>>>>>>>>> get the access tokens in
the elastic search index. Hoping for the solution
>>>>>>>>>>>>>>>> asap.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Thanks and Regards,
>>>>>>>>>>>>>>>> Nikita
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>
>>>
>>
>

Mime
View raw message