manifoldcf-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Nikita Ahuja <nik...@smartshore.nl>
Subject Re: Problem in fetching the access tokens from Active Directory in Elastic Search index for FileSystem Connector
Date Mon, 05 Feb 2018 08:21:17 GMT
Hi Karl,

I have done the configuration of the job as per the *End User Documentation*.
Still, I am getting "no security" value in the access tokens in the elastic
search index (output of my job).

[image: Inline image 2]


This is the Output connector for Elastic Search. Is there anything needs to
added here?

[image: Inline image 1]



Also, the security is also enabled.

[image: Inline image 3]


I am not able to understand where is the actual problem?


Thanks and Regards,
Nikita




On Fri, Feb 2, 2018 at 7:53 PM, Karl Wright <daddywri@gmail.com> wrote:

> Hi Nikita,
>
> If you are seeing *no* tokens indexed then I would check your output
> connection and your index configuration.  Custom fields need to be added to
> Solr for these tokens to be indexed, if you are using Solr.  There are
> instructions in the online documentation on what to add and how to
> configure them.
>
> The other possibility is that you have disabled security for you
> Sharepoint job in the "Security" tab.
>
> Karl
>
>
> On Fri, Feb 2, 2018 at 8:48 AM, Nikita Ahuja <nikita@smartshore.nl> wrote:
>
>> Hi Karl,
>>
>>
>> Yes, you are right I am able to fetch the files  but the access tokens
>> indexed with the files are not included what is required. And I am using
>> the ManifoldCF security model not any custom. Everything is fine but only
>> problem is with the token.
>> And am not sure what is the reason behind that: is it because the
>> Sharepoint site which is used is not with *HTTPS* protocol or any other
>> reason?
>>
>>
>> Thanks and Regards,
>> Nikita
>>
>> On Fri, Feb 2, 2018 at 7:05 PM, Karl Wright <daddywri@gmail.com> wrote:
>>
>>> I am not sure I understand -- are you saying that you can fetch the
>>> files but the access tokens indexed with the files do not include what you
>>> require? And that the reason you require it done a certain way is because
>>> you intend to apply security in a manner of your own design?
>>>
>>> The access tokens that are indexed with files in SharePoint are what are
>>> needed to enforce security using the ManifoldCF Sharepoint authorities, no
>>> more and no less.  The authorities map an individual user to the access
>>> tokens needed for that user, which include tokens for the groups as well as
>>> for the user name.  The tokens indexed with the files do NOT include tokens
>>> generated by mapping groups to individual users -- the only tokens indexed
>>> for a file are for the groups and for any individual users specified
>>> explicitly.
>>>
>>> I strongly suggest you attempt to apply security fully using the
>>> ManifoldCF model rather than trying to do something custom.
>>>
>>> Karl
>>>
>>>
>>> On Fri, Feb 2, 2018 at 8:04 AM, Nikita Ahuja <nikita@smartshore.nl>
>>> wrote:
>>>
>>>> Hi Karl,
>>>>
>>>> Thanks a lot for your help.It is working fine now.. I am able to fetch
>>>> all the documents in the path but there is a requirement to get the access
>>>> tokens for the files. But,it shows the authority group name not any details
>>>> related to the users who all have access for the document accessed.
>>>>
>>>> Please guide for that.
>>>>
>>>>
>>>> [image: Inline image 1]
>>>>
>>>>
>>>> Thanks and Regards,
>>>> Nikita
>>>>
>>>> On Wed, Jan 31, 2018 at 5:55 PM, Karl Wright <daddywri@gmail.com>
>>>> wrote:
>>>>
>>>>> Clearly you are unable to fetch document ACLs from SharePoint.  This
>>>>> has nothing to do with whether your Active Directory is properly
>>>>> configured, and has more to do with the MCPermissions plugin.
>>>>>
>>>>> When you view the SharePoint repository connection, does it say
>>>>> "Connection working"?  If not, you need the figure out why.
>>>>>
>>>>> Here are some possibilities:
>>>>> (1) The plugin was properly installed but with the wrong permissions.
>>>>> You must be logged in as an Admin to install it properly.  If that's
not
>>>>> done it will mostly work but not be able to retrieve the permissions.
>>>>> (2) You may have selected the wrong version of Sharepoint in the
>>>>> repository connection.  Make sure you select the right one or, once again,
>>>>> you won't be able to retrieve permissions.
>>>>>
>>>>> The documentation says this: "Select your SharePoint server version
>>>>> from the pulldown. If you do not select the correct server version, your
>>>>> documents may either be indexed with insufficient security protection,
or
>>>>> you may not be able to index any documents. Check with your SharePoint
>>>>> system administrator if you are not sure what to select."
>>>>>
>>>>> Please let me know whether this helps.  Thanks!
>>>>> Karl
>>>>>
>>>>>
>>>>> On Wed, Jan 31, 2018 at 6:51 AM, Nikita Ahuja <nikita@smartshore.nl>
>>>>> wrote:
>>>>>
>>>>>> Hi Karl,
>>>>>>
>>>>>> I have checked the log Files but there is issue with the ACLs but
>>>>>> there is Active Directory already connected with Sharepoint site.
And it is
>>>>>> not possible to pass the Access tokens in the connector. Please provide
the
>>>>>> suggestion for the problem.
>>>>>>
>>>>>> [image: Inline image 1]
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> Thanks and Regards,
>>>>>> Nikita
>>>>>>
>>>>>> On Mon, Jan 29, 2018 at 6:55 PM, Karl Wright <daddywri@gmail.com>
>>>>>> wrote:
>>>>>>
>>>>>>> I don't see any path rules there.  That will clearly not work.
>>>>>>>
>>>>>>> If you want confirmation of this, you can turn on connector
>>>>>>> debugging (set properties.xml property org.apache.manifoldcf.connectors
>>>>>>> to "DEBUG"), look at the log output.  Every time the SharePoint
connector
>>>>>>> makes a decision to not include something, it logs it.
>>>>>>>
>>>>>>> Karl
>>>>>>>
>>>>>>> On Mon, Jan 29, 2018 at 8:07 AM, Nikita Ahuja <nikita@smartshore.nl>
>>>>>>> wrote:
>>>>>>>
>>>>>>>> Hi,
>>>>>>>>
>>>>>>>> I have checked the documentation and the path configuration
as
>>>>>>>> well. But still not able to get any output.
>>>>>>>>
>>>>>>>> [image: Inline image 1]
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> and the metadata path given is:
>>>>>>>> [image: Inline image 2]
>>>>>>>>
>>>>>>>> Thanks and Regards,
>>>>>>>> Nikita
>>>>>>>>
>>>>>>>> On Mon, Jan 29, 2018 at 4:45 PM, Karl Wright <daddywri@gmail.com>
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>>> Hi,
>>>>>>>>>
>>>>>>>>> I think your path configuration is incorrect.  See the
end-user
>>>>>>>>> documentation.  It's a little tricky because you must
specify a matching
>>>>>>>>> rule for both files and paths.
>>>>>>>>>
>>>>>>>>> Karl
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> On Mon, Jan 29, 2018 at 4:42 AM, Nikita Ahuja <
>>>>>>>>> nikita@smartshore.nl> wrote:
>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>    - Hi Karl,
>>>>>>>>>>
>>>>>>>>>> There is another query related to Sharepoint 2013
Connector. I am
>>>>>>>>>> trying to crawl the files of Sharepoint On Premise
site. But I am not able
>>>>>>>>>> to get anything in the output either in ElasticSearch
or in FileShare.
>>>>>>>>>> I have followed given steps:
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>    - Created an Authority group for the Active Directory
and
>>>>>>>>>>    same is used in Sharepoint Repository Connection.
>>>>>>>>>>
>>>>>>>>>> [image: Inline image 2]
>>>>>>>>>>
>>>>>>>>>>    - Installed the Sharepoint 2013 MCPermission Plugin
and is
>>>>>>>>>>    accessible on the site.
>>>>>>>>>>
>>>>>>>>>> [image: Inline image 3]
>>>>>>>>>>
>>>>>>>>>>    - Still not able to access any document from the
selected
>>>>>>>>>>    folder of Sharepoint Site.
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> Please guide me accordingly.
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> Thanks and Regards,
>>>>>>>>>> Nikita
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> On Thu, Jan 25, 2018 at 6:46 PM, Nikita Ahuja <
>>>>>>>>>> nikita@smartshore.nl> wrote:
>>>>>>>>>>
>>>>>>>>>>> Thanks Karl,
>>>>>>>>>>>
>>>>>>>>>>> It is working fine now and Also I am able to
get the access
>>>>>>>>>>> tokens in the elastic search index.
>>>>>>>>>>>
>>>>>>>>>>> Thanks a lot for your valuable suggestions.
>>>>>>>>>>>
>>>>>>>>>>> With Regards,
>>>>>>>>>>> Nikita
>>>>>>>>>>>
>>>>>>>>>>> On Thu, Jan 25, 2018 at 4:42 PM, Karl Wright
<daddywri@gmail.com
>>>>>>>>>>> > wrote:
>>>>>>>>>>>
>>>>>>>>>>>> The __nosecurity__ token is generated by
the Elastic Search
>>>>>>>>>>>> Connector.  It is produced (correctly) when
no acls reach the connector.
>>>>>>>>>>>> This is why I asked you about the repository
connection, because that
>>>>>>>>>>>> connection is not producing any ACLs.
>>>>>>>>>>>>
>>>>>>>>>>>> If you are crawling files in a file system,
you must use the
>>>>>>>>>>>> JCIFS connector if you want ACLs to be generated.
 The FileSystem connector
>>>>>>>>>>>> does not generate ACLs.
>>>>>>>>>>>>
>>>>>>>>>>>> Karl
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> On Thu, Jan 25, 2018 at 3:56 AM, Nikita Ahuja
<
>>>>>>>>>>>> nikita@smartshore.nl> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>>> Hi Karl,
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> I have tried for ElasticSearch version
1.3.1 and 5.6.1 but
>>>>>>>>>>>>> still getting this " __nosecurity__"
token and for Repository
>>>>>>>>>>>>> connection  File Sytem is used and for
Transformation  "Tika Content
>>>>>>>>>>>>> Extractor" and Metadata adjuster Connector
is used.
>>>>>>>>>>>>>
>>>>>>>>>>>>> [image: Inline image 1]
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> Is there any other thing which I need
to perform??
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> Thanks and Regards,
>>>>>>>>>>>>> Nikita
>>>>>>>>>>>>>
>>>>>>>>>>>>> On Thu, Jan 25, 2018 at 2:13 PM, Karl
Wright <
>>>>>>>>>>>>> daddywri@gmail.com> wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>>> Hi Nikita,
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> You are getting the __nosecurity__
token value transmitted
>>>>>>>>>>>>>> and stored, which means your Elastic
Search setup is probably reasonable.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Can you give us details about your
pipeline?  What repository
>>>>>>>>>>>>>> connector is this?
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Karl
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> On Thu, Jan 25, 2018 at 1:49 AM,
Nikita Ahuja <
>>>>>>>>>>>>>> nikita@smartshore.nl> wrote:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Hello,
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> I am using ManifoldCF 2.8.1 and
trying to access the files
>>>>>>>>>>>>>>> from FileShare configured with
the Active Directory and get the data in
>>>>>>>>>>>>>>> elastic search index.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>  For this steps followed are:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>    - An Authority Group is created
under which Authority
>>>>>>>>>>>>>>>    Connection and Repository
connection are created.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> [image: Inline image 3]
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>    - And the elastic search is
created as an Output
>>>>>>>>>>>>>>>    Connector and a job is created
for the connection in which a transformation
>>>>>>>>>>>>>>>    and metadata connector is
also linked in the
>>>>>>>>>>>>>>>    - Which is giving all the
output except the access token
>>>>>>>>>>>>>>>    values.
>>>>>>>>>>>>>>>    - [image: Inline image 1]
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Please guide to take appropriate
steps for it so that I can
>>>>>>>>>>>>>>> get the access tokens in the
elastic search index. Hoping for the solution
>>>>>>>>>>>>>>> asap.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Thanks and Regards,
>>>>>>>>>>>>>>> Nikita
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>
>>>
>>
>

Mime
View raw message