Return-Path: X-Original-To: apmail-manifoldcf-user-archive@www.apache.org Delivered-To: apmail-manifoldcf-user-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 17DAD10ADB for ; Fri, 22 Nov 2013 06:19:35 +0000 (UTC) Received: (qmail 44200 invoked by uid 500); 22 Nov 2013 06:19:33 -0000 Delivered-To: apmail-manifoldcf-user-archive@manifoldcf.apache.org Received: (qmail 44136 invoked by uid 500); 22 Nov 2013 06:19:28 -0000 Mailing-List: contact user-help@manifoldcf.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@manifoldcf.apache.org Delivered-To: mailing list user@manifoldcf.apache.org Received: (qmail 44128 invoked by uid 99); 22 Nov 2013 06:19:26 -0000 Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 22 Nov 2013 06:19:26 +0000 X-ASF-Spam-Status: No, hits=1.5 required=5.0 tests=HTML_MESSAGE,RCVD_IN_DNSWL_LOW,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (nike.apache.org: domain of parkinson.will@gmail.com designates 74.125.82.180 as permitted sender) Received: from [74.125.82.180] (HELO mail-we0-f180.google.com) (74.125.82.180) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 22 Nov 2013 06:19:19 +0000 Received: by mail-we0-f180.google.com with SMTP id u56so698797wes.39 for ; Thu, 21 Nov 2013 22:18:58 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=UP8GfAmBMaAW3nn8iAdeNABl1r5kkTBM0X791lfWsEs=; b=odDtHt8R3ieLCw96BvvcllJXDPRol/YUo+G2k/BgeAVuvy7/91D2LlXFzAZx6stpoD duGMjqFjmLeohXQujb6iGhQ3I5RqxlTltg6raEI0mA/fxHdZhoID3mAwgrL6526H/E3u vkrUecNy4L9C9iJeokUbYPK150BD2gZ1xgHIT2BofUXltoeY48o7VxPgATX4f1rCO2Xd pDM7EouZYfIlU61Ncg+xLKAwo6fASLqwF28ZgINS7NoVwZI8Ack/53H/d1ib5PR5OLzN lxl72/kX7mM3puiy/0yKVbb4gprlEBI6gXjsHquKURWO54l+qOyQ6zVXdtWhnaX4f/YD H+xQ== MIME-Version: 1.0 X-Received: by 10.180.76.196 with SMTP id m4mr1071669wiw.59.1385101138773; Thu, 21 Nov 2013 22:18:58 -0800 (PST) Received: by 10.180.107.132 with HTTP; Thu, 21 Nov 2013 22:18:58 -0800 (PST) In-Reply-To: References: <-8185508655186956523@unknownmsgid> Date: Fri, 22 Nov 2013 16:18:58 +1000 Message-ID: Subject: Re: Sharepoint SID extraction for groups From: Will Parkinson To: Karl Wright Cc: "user@manifoldcf.apache.org" Content-Type: multipart/alternative; boundary=f46d043c7f065ad84204ebbdfd49 X-Virus-Checked: Checked by ClamAV on apache.org --f46d043c7f065ad84204ebbdfd49 Content-Type: text/plain; charset=ISO-8859-1 Thanks for that Karl, it sounds like a good way forward. I have built trunk and installed the 1.5-dev version and started the setup, but have found an issue with the Sharepoint Native connection I am getting this error Connection status: Accessing site failed with unexpected SharePoint error code 0x80131600: User cannot be found. What sort of Sharepoint credentials do i need for the Native connection? Cheers, Will On Fri, Nov 22, 2013 at 12:39 AM, Karl Wright wrote: > Hi Will, > > I've committed to trunk the following: > > - Two new authorities: SharePoint/Native and SharePoint/AD > - A revised SharePoint connector, which has the ability to EITHER use the > legacy AD authority, or the native SharePoint family of authorities. > > If you are willing to experiment with trunk code, I think you will find > that using SharePoint native authorities will solve your long-list-of-SIDs > issue. If you use both the SharePoint/Native authority and the > SharePoint/AD authority under the same authority group that the repository > connection uses, in theory it should support full Claim Space auth. I say > "in theory" because I have not had the opportunity to test it yet in an > actual environment. I'd love to have that chance. > > Please let me know if you are willing to work collaboratively on finishing > this off. I think it would be far better to take this route than continue > to hack away at 1.4 code or earlier. > > What do you think? > > Karl > > > > > On Thu, Nov 21, 2013 at 9:23 AM, Will Parkinson wrote: > >> Hi Karl, >> >> This looks like its a postgres 9.1 issue, i downgraded to postgres 8.4 >> and it's no longer an issue. >> >> Just back to the claims based authentication, i ended up writing a class >> that extracts the missing SID's for sharepoint groups from AD which works >> perfectly fine in itself, but the SID lists are huge (sometime in excess of >> 1.5MB) which is causing an issue with database slowness. Inserting and >> trawling through tables with large amounts of SID's stored in them seems to >> be a problem. >> >> I am just looking for a solution and testing a few things, including >> storing the SID's in a file on the server and "attaching" them before they >> go out of the custom output connector we have built. >> >> Is there an easy way in the SPSProxyHelper.java class that i can get the >> full sharepoint URL of the page being processed? >> >> Cheers, >> >> Will >> >> >> On Mon, Nov 18, 2013 at 2:54 AM, Karl Wright wrote: >> >>> Hi Will, >>> >>> I looked at the pivot exception, but it seems like it *is* detecting >>> that it should retry the transaction, and is indeed retrying. This is >>> expected behavior. You did not include the beginning of the message; if it >>> was DEBUG or WARNING I would be comfortable that it was doing the right >>> thing. >>> >>> Somewhere else, though, there may well be an actual database ERROR that >>> is causing the system to get hung. This will show up as an ERROR in the >>> log, and when you do a thread dump, all the worker threads will be waiting >>> on something in WorkerResetManager. Could you include more of the log so >>> that I can have a look at this? >>> >>> Thanks, >>> Karl >>> >>> >>> >>> On Sat, Nov 16, 2013 at 2:06 PM, Karl Wright wrote: >>> >>>> Hi will, >>>> The long running query is not fatal - it is just a warning. >>>> >>>> The very-long sid list requires a SharePoint authority, as discussed. >>>> >>>> The pivot error sounds like it is something that can be addressed >>>> though. Please create a ticket and put the full exception into it, >>>> and I will look at it either tomorrow or Monday. >>>> >>>> Thanks, >>>> Karl >>>> >>>> Sent from my Windows Phone >>>> >>>> -----Original Message----- >>>> From: Will Parkinson >>>> Sent: 11/16/2013 10:10 AM >>>> To: user@manifoldcf.apache.org >>>> Subject: Re: Sharepoint SID extraction for groups >>>> >>>> >>>> >>>> >>>> >>>> >>>> >>>> >>>> Hi Karl, >>>> >>>> >>>> Yeah that seems to be be case, to get ManifoldCF to work in my case i >>>> just created a separate class to obtain all the user SID's directly >>>> from AD if the group assigned in Sharepoint is an AD group. This >>>> seems to work fine for now, but it seems to be causing a few database >>>> issues. >>>> >>>> First of all, some of the SID lists are up to 1.5MB, which seems to be >>>> causing the carrydown table to become huge. I am also getting errors >>>> like >>>> >>>> 1C159E0: ERROR: could not serialize access due to read/write >>>> dependencies among transactions >>>> Detail: Reason code: Canceled on identification as a pivot, during >>>> conflict in checking. >>>> Hint: The transaction might succeed if retried.; sleeping for 56816 ms >>>> org.apache.manifoldcf.core.interfaces.ManifoldCFException: ERROR: >>>> could not serialize access due to read/write dependencies among >>>> transactions >>>> Detail: Reason code: Canceled on identification as a pivot, during >>>> conflict in checking. >>>> Hint: The transaction might succeed if retried. >>>> at >>>> org.apache.manifoldcf.core.database.DBInterfacePostgreSQL.reinterpretException(DBInterfacePostgreSQL.java:622) >>>> at >>>> org.apache.manifoldcf.core.database.DBInterfacePostgreSQL.performModification(DBInterfacePostgreSQL.java:651) >>>> at >>>> org.apache.manifoldcf.core.database.DBInterfacePostgreSQL.performInsert(DBInterfacePostgreSQL.java:187) >>>> at >>>> org.apache.manifoldcf.core.database.BaseTable.performInsert(BaseTable.java:68) >>>> at >>>> org.apache.manifoldcf.crawler.jobs.Carrydown.recordCarrydownDataMultiple(Carrydown.java:343) >>>> at >>>> org.apache.manifoldcf.crawler.jobs.JobManager.addDocuments(JobManager.java:4174) >>>> at >>>> org.apache.manifoldcf.crawler.system.WorkerThread$ProcessActivity.processDocumentReferences(WorkerThread.java:2017) >>>> at >>>> org.apache.manifoldcf.crawler.system.WorkerThread$ProcessActivity.flush(WorkerThread.java:1948) >>>> at >>>> org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThread.java:562) >>>> Caused by: org.postgresql.util.PSQLException: ERROR: could not >>>> serialize access due to read/write dependencies among transactions >>>> Detail: Reason code: Canceled on identification as a pivot, during >>>> conflict in checking. >>>> Hint: The transaction might succeed if retried. >>>> at >>>> org.postgresql.core.v3.QueryExecutorImpl.receiveErrorResponse(QueryExecutorImpl.java:2102) >>>> at >>>> org.postgresql.core.v3.QueryExecutorImpl.processResults(QueryExecutorImpl.java:1835) >>>> at >>>> org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:257) >>>> at >>>> org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Statement.java:500) >>>> at >>>> org.postgresql.jdbc2.AbstractJdbc2Statement.executeWithFlags(AbstractJdbc2Statement.java:388) >>>> at >>>> org.postgresql.jdbc2.AbstractJdbc2Statement.executeUpdate(AbstractJdbc2Statement.java:334) >>>> at >>>> org.apache.manifoldcf.core.database.Database.execute(Database.java:883) >>>> at >>>> org.apache.manifoldcf.core.database.Database$ExecuteQueryThread.run(Database.java:683) >>>> >>>> And then i eventually get an error like this >>>> >>>> WARN 2013-11-17 00:41:09,058 (Finisher thread) - Found a long-running >>>> query (77260 ms): [SELECT id FROM jobs WHERE status IN (?,?,?,?,?) FOR >>>> UPDATE] >>>> WARN 2013-11-17 00:41:09,059 (Finisher thread) - Parameter 0: 'A' >>>> WARN 2013-11-17 00:41:09,059 (Finisher thread) - Parameter 1: 'W' >>>> WARN 2013-11-17 00:41:09,059 (Finisher thread) - Parameter 2: 'R' >>>> WARN 2013-11-17 00:41:09,059 (Finisher thread) - Parameter 3: 'O' >>>> WARN 2013-11-17 00:41:09,059 (Finisher thread) - Parameter 4: 'U' >>>> WARN 2013-11-17 00:41:09,060 (Finisher thread) - Plan: LockRows >>>> (cost=0.00..3.34 rows=5 width=14) (actual time=0.026..0.027 rows=1 >>>> loops=1) >>>> WARN 2013-11-17 00:41:09,060 (Finisher thread) - Plan: -> Seq >>>> Scan on jobs (cost=0.00..3.29 rows=5 width=14) (actual >>>> time=0.024..0.024 rows=1 loops=1) >>>> WARN 2013-11-17 00:41:09,060 (Finisher thread) - Plan: >>>> Filter: (status = ANY ('{A,W,R,O,U}'::bpchar[])) >>>> WARN 2013-11-17 00:41:09,060 (Finisher thread) - Plan: Rows >>>> Removed by Filter: 17 >>>> WARN 2013-11-17 00:41:09,060 (Finisher thread) - Plan: Total runtime: >>>> 0.058 ms >>>> WARN 2013-11-17 00:41:09,060 (Finisher thread) - >>>> >>>> And then the update stops completely, even though the status on the >>>> "Status and job management page" is still set as "running". Do you >>>> have any ideas on how i can fix this? >>>> >>>> I am doing some research at the moment on the best way to store >>>> permissions information without storing 100's of SID's. >>>> >>>> Cheers, >>>> >>>> Will >>>> >>>> >>>> >>>> >>>> On Wed, Nov 6, 2013 at 11:42 PM, Karl Wright >>>> wrote: >>>> >>>> >>>> I should also add that, as far as ActiveDirectory groups go, my >>>> understanding is that in non-Claim-Space versions of SharePoint, >>>> there's a SharePoint group created for each AD group. So a SharePoint >>>> user will belong to some native SharePoint groups, but also to some >>>> "mirrored" SharePoint groups that are created because of the user's >>>> group relationships in AD. >>>> >>>> Claim Space seems to change this in the following way: SharePoint >>>> groups no longer mirror AD groups. Instead, the Claim Space >>>> authorization tokens implicitly describe the relationships. So you >>>> have to talk to both SharePoint AND AD in order to fully understand >>>> what documents in SharePoint are authorized for what users. >>>> >>>> Karl >>>> >>>> >>>> >>>> >>>> >>>> >>>> >>>> >>>> On Wed, Nov 6, 2013 at 8:37 AM, Karl Wright wrote: >>>> >>>> >>>> >>>> >>>> >>>> >>>> >>>> >>>> >>>> Hi Will, >>>> >>>> >>>> >>>> The current connector indeed maps SharePoint groups to individual user >>>> SIDs. That is not terribly scalable, and it is one reason why I've >>>> created dedicated SharePoint authorities in the CONNECTORS-754-2 >>>> branch, so that we can authorize documents at a group level. >>>> >>>> >>>> I've also done considerable research on the ClaimSpace security model. >>>> Supporting it fully has required some modifications to the basic >>>> authorization model that ManifoldCF uses to tie documents to >>>> authorities. This basic work is done and is now part of trunk as >>>> well. And the documentation has been updated to describe the revised >>>> authorization model. >>>> >>>> If you want to try working with the CONNECTORS-754-2 branch, I'd be >>>> very happy to interact with you to iron out any problems. What you >>>> will need to do if you try it out is the following: >>>> >>>> (1) Create an authority group for your SharePoint instance >>>> (2) Create a "SharePoint/Native" authority tied to that authority group >>>> (3) If this is a claim-space SharePoint instance, then also create a >>>> "SharePoint/Active Directory" authority tied to the same authority >>>> group >>>> (4) Create your SharePoint repository connection, making sure to >>>> select "Native" mode >>>> >>>> The implementation is currently the best I can do in the absence of a >>>> full-blown Claim Space instance. Even so, there are still questions >>>> in my mind that, if I could solve them, would help clarify the >>>> implementation. For example, what "Role Definitions" do - are they >>>> essentially just another form of group? And, whether it is better to >>>> use a user, group, or role definition's name for an access token, or >>>> the ID? Perhaps you can clarify a bit, I don't know... >>>> >>>> >>>> Thanks, >>>> Karl >>>> >>>> >>>> >>>> >>>> >>>> >>>> >>>> On Wed, Nov 6, 2013 at 8:14 AM, Will Parkinson < >>>> parkinson.will@gmail.com> wrote: >>>> >>>> >>>> >>>> >>>> >>>> >>>> Hello, >>>> >>>> >>>> I am just wondering how the extraction of the groups permissions works >>>> for the sharepoint connector. From what I can see, it seems that the >>>> group is determined via the MCPermissions.asmx web service and then >>>> each user in that group is iterated over and the SID for those users >>>> are extracted. >>>> >>>> Is this the case? If so, are groups created in Sharepoint and AD >>>> groups treated the same way? >>>> >>>> Cheers, >>>> >>>> Will >>>> >>> >>> >> > --f46d043c7f065ad84204ebbdfd49 Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable
Thanks for that Karl, it sounds like a= good way forward.

I have built trunk and installed the 1.5-dev version and started the=20 setup, but have found an issue with the Sharepoint Native connection
I am getting this error


What sort of Sharepoint credentials do i need for the Native c= onnection?

Cheers,

Will


<= div class=3D"gmail_quote">On Fri, Nov 22, 2013 at 12:39 AM, Karl Wright <= daddywri@gmail.com> wrote:
Hi= Will,

I've committed to trunk the following:

- Two new authorities: SharePoint/Native and SharePoint/AD
- = A revised SharePoint connector, which has the ability to EITHER use the leg= acy AD authority, or the native SharePoint family of authorities.

If you are willing to experiment with trunk code, I think you wil= l find that using SharePoint native authorities will solve your long-list-o= f-SIDs issue.=A0 If you use both the SharePoint/Native authority and the Sh= arePoint/AD authority under the same authority group that the repository co= nnection uses, in theory it should support full Claim Space auth.=A0 I say = "in theory" because I have not had the opportunity to test it yet= in an actual environment.=A0 I'd love to have that chance.

Please let me know if you are willing to work collaboratively on = finishing this off.=A0 I think it would be far better to take this route th= an continue to hack away at 1.4 code or earlier.

What do you t= hink?

Karl




On Thu, Nov 21, 2013 at 9:23 AM, Will P= arkinson <parkinson.will@gmail.com> wrote:
Hi Karl= ,

This looks like its a postgres 9.1 issue, i downgraded to postgres= 8.4 and it's no longer an issue.

Just back to the claims based authentication, i ended up writing = a class that extracts the missing SID's for sharepoint groups from AD w= hich works perfectly fine in itself, but the SID lists are huge (sometime i= n excess of 1.5MB) which is causing an issue with database slowness.=A0 Ins= erting and trawling through tables with large amounts of SID's stored i= n them seems to be a problem.

I am just looking for a solution and testing a few things, includ= ing storing the SID's in a file on the server and "attaching"= them before they go out of the custom output connector we have built.

Is there an easy way in the SPSProxyHelper.java class that i can = get the full sharepoint URL of the page being processed?

Cheer= s,

Will


On Mon, Nov 18, 2013 at 2:54 AM, Karl Wright <daddywri@gmail.com>= wrote:
Hi Will,

I looked at the pivot exce= ption, but it seems like it *is* detecting that it should retry the transac= tion, and is indeed retrying.=A0 This is expected behavior.=A0 You did not = include the beginning of the message; if it was DEBUG or WARNING I would be= comfortable that it was doing the right thing.

Somewhere else, though, there may well be an actual database ERRO= R that is causing the system to get hung.=A0 This will show up as an ERROR = in the log, and when you do a thread dump, all the worker threads will be w= aiting on something in WorkerResetManager.=A0 Could you include more of the= log so that I can have a look at this?

Thanks,
Karl


<= br>
On Sat, Nov 16, 2013 at 2:06 PM, Karl Wright = <daddywri@gmail.com> wrote:
Hi will,
The long running query is not fatal - it is just a warning.

The very-long sid list requires a SharePoint authority, as discussed.

The pivot error sounds like it is something that can be addressed
though. =A0Please create a ticket and put the full exception into it,
and I will look at it either tomorrow or Monday.

Thanks,
Karl

Sent from my Windows Phone

-----Original Message-----
From: Will Parkinson
Sent: 11/16/2013 10:10 AM
To: user@ma= nifoldcf.apache.org
Subject: Re: Sharepoint SID extraction for groups








Hi Karl,


Yeah that seems to be be case, to get ManifoldCF to work in my case i
just created a separate class to obtain all the user SID's directly
from AD if the group assigned in Sharepoint is an AD group. =A0This
seems to work fine for now, but it seems to be causing a few database
issues.

First of all, some of the SID lists are up to 1.5MB, which seems to be
causing the carrydown table to become huge. =A0I am also getting errors
like

1C159E0: ERROR: could not serialize access due to read/write
dependencies among transactions
=A0 =A0Detail: Reason code: Canceled on identification as a pivot, during conflict in checking.
=A0 Hint: The transaction might succeed if retried.; sleeping for 56816 ms<= br> org.apache.manifoldcf.core.interfaces.ManifoldCFException: ERROR:
could not serialize access due to read/write dependencies among
transactions
=A0 =A0Detail: Reason code: Canceled on identification as a pivot, during conflict in checking.
=A0 Hint: The transaction might succeed if retried.
=A0 =A0 =A0 =A0 at org.apache.manifoldcf.core.database.DBInterfacePostgreSQ= L.reinterpretException(DBInterfacePostgreSQL.java:622)
=A0 =A0 =A0 =A0 =A0at org.apache.manifoldcf.core.database.DBInterfacePostgr= eSQL.performModification(DBInterfacePostgreSQL.java:651)
=A0 =A0 =A0 =A0 at org.apache.manifoldcf.core.database.DBInterfacePostgreSQ= L.performInsert(DBInterfacePostgreSQL.java:187)
=A0 =A0 =A0 =A0 =A0at org.apache.manifoldcf.core.database.BaseTable.perform= Insert(BaseTable.java:68)
=A0 =A0 =A0 =A0 at org.apache.manifoldcf.crawler.jobs.Carrydown.recordCarry= downDataMultiple(Carrydown.java:343)
=A0 =A0 =A0 =A0 at org.apache.manifoldcf.crawler.jobs.JobManager.addDocumen= ts(JobManager.java:4174)
=A0 =A0 =A0 =A0 =A0at org.apache.manifoldcf.crawler.system.WorkerThread$Pro= cessActivity.processDocumentReferences(WorkerThread.java:2017)
=A0 =A0 =A0 =A0 at org.apache.manifoldcf.crawler.system.WorkerThread$Proces= sActivity.flush(WorkerThread.java:1948)
=A0 =A0 =A0 =A0 =A0at org.apache.manifoldcf.crawler.system.WorkerThread.run= (WorkerThread.java:562)
Caused by: org.postgresql.util.PSQLException: ERROR: could not
serialize access due to read/write dependencies among transactions
=A0 =A0Detail: Reason code: Canceled on identification as a pivot, during conflict in checking.
=A0 Hint: The transaction might succeed if retried.
=A0 =A0 =A0 =A0 at org.postgresql.core.v3.QueryExecutorImpl.receiveErrorRes= ponse(QueryExecutorImpl.java:2102)
=A0 =A0 =A0 =A0 =A0at org.postgresql.core.v3.QueryExecutorImpl.processResul= ts(QueryExecutorImpl.java:1835)
=A0 =A0 =A0 =A0 at org.postgresql.core.v3.QueryExecutorImpl.execute(QueryEx= ecutorImpl.java:257)
=A0 =A0 =A0 =A0 at org.postgresql.jdbc2.AbstractJdbc2Statement.execute(Abst= ractJdbc2Statement.java:500)
=A0 =A0 =A0 =A0 =A0at org.postgresql.jdbc2.AbstractJdbc2Statement.executeWi= thFlags(AbstractJdbc2Statement.java:388)
=A0 =A0 =A0 =A0 at org.postgresql.jdbc2.AbstractJdbc2Statement.executeUpdat= e(AbstractJdbc2Statement.java:334)
=A0 =A0 =A0 =A0 at org.apache.manifoldcf.core.database.Database.execute(Dat= abase.java:883)
=A0 =A0 =A0 =A0 =A0at org.apache.manifoldcf.core.database.Database$ExecuteQ= ueryThread.run(Database.java:683)

And then i eventually get an error like this

=A0WARN 2013-11-17 00:41:09,058 (Finisher thread) - Found a long-running query (77260 ms): [SELECT id FROM jobs WHERE status IN (?,?,?,?,?) FOR
UPDATE]
=A0 WARN 2013-11-17 00:41:09,059 (Finisher thread) - =A0 Parameter 0: '= A'
=A0WARN 2013-11-17 00:41:09,059 (Finisher thread) - =A0 Parameter 1: 'W= '
=A0WARN 2013-11-17 00:41:09,059 (Finisher thread) - =A0 Parameter 2: 'R= '
=A0 WARN 2013-11-17 00:41:09,059 (Finisher thread) - =A0 Parameter 3: '= O'
=A0WARN 2013-11-17 00:41:09,059 (Finisher thread) - =A0 Parameter 4: 'U= '
=A0WARN 2013-11-17 00:41:09,060 (Finisher thread) - =A0Plan: LockRows
(cost=3D0.00..3.34 rows=3D5 width=3D14) (actual time=3D0.026..0.027 rows=3D= 1
loops=3D1)
=A0 WARN 2013-11-17 00:41:09,060 (Finisher thread) - =A0Plan: =A0 -> =A0= Seq
Scan on jobs =A0(cost=3D0.00..3.29 rows=3D5 width=3D14) (actual
time=3D0.024..0.024 rows=3D1 loops=3D1)
=A0WARN 2013-11-17 00:41:09,060 (Finisher thread) - =A0Plan:
Filter: (status =3D ANY ('{A,W,R,O,U}'::bpchar[]))
=A0 WARN 2013-11-17 00:41:09,060 (Finisher thread) - =A0Plan: =A0 =A0 =A0 = =A0 Rows
Removed by Filter: 17
=A0WARN 2013-11-17 00:41:09,060 (Finisher thread) - =A0Plan: Total runtime:= 0.058 ms
=A0WARN 2013-11-17 00:41:09,060 (Finisher thread) -

And then the update stops completely, even though the status on the
"Status and job management page" is still set as "running&qu= ot;. =A0Do you
have any ideas on how i can fix this?

I am doing some research at the moment on the best way to store
permissions information without storing 100's of SID's.

Cheers,

Will




On Wed, Nov 6, 2013 at 11:42 PM, Karl Wright <daddywri@gmail.com> wrote:


I should also add that, as far as ActiveDirectory groups go, my
understanding is that in non-Claim-Space versions of SharePoint,
there's a SharePoint group created for each AD group. =A0So a SharePoin= t
user will belong to some native SharePoint groups, but also to some
"mirrored" SharePoint groups that are created because of the user= 's
group relationships in AD.

Claim Space seems to change this in the following way: SharePoint
groups no longer mirror AD groups. =A0Instead, the Claim Space
authorization tokens implicitly describe the relationships. =A0So you
have to talk to both SharePoint AND AD in order to fully understand
what documents in SharePoint are authorized for what users.

Karl








On Wed, Nov 6, 2013 at 8:37 AM, Karl Wright <daddywri@gmail.com> wrote:









Hi Will,



The current connector indeed maps SharePoint groups to individual user
SIDs. =A0That is not terribly scalable, and it is one reason why I've created dedicated SharePoint authorities in the CONNECTORS-754-2
branch, so that we can authorize documents at a group level.


I've also done considerable research on the ClaimSpace security model.<= br> =A0Supporting it fully has required some modifications to the basic
authorization model that ManifoldCF uses to tie documents to
authorities. =A0This basic work is done and is now part of trunk as
well. =A0And the documentation has been updated to describe the revised
authorization model.

If you want to try working with the CONNECTORS-754-2 branch, I'd be
very happy to interact with you to iron out any problems. =A0What you
will need to do if you try it out is the following:

(1) Create an authority group for your SharePoint instance
(2) Create a "SharePoint/Native" authority tied to that authority= group
(3) If this is a claim-space SharePoint instance, then also create a
"SharePoint/Active Directory" authority tied to the same authorit= y
group
(4) Create your SharePoint repository connection, making sure to
select "Native" mode

The implementation is currently the best I can do in the absence of a
full-blown Claim Space instance. =A0Even so, there are still questions
in my mind that, if I could solve them, would help clarify the
implementation. =A0For example, what "Role Definitions" do - are = they
essentially just another form of group? =A0And, whether it is better to
use a user, group, or role definition's name for an access token, or the ID? =A0Perhaps you can clarify a bit, I don't know...


Thanks,
Karl







On Wed, Nov 6, 2013 at 8:14 AM, Will Parkinson <parkinson.will@gmail.com> wrot= e:






Hello,


I am just wondering how the extraction of the groups permissions works
for the sharepoint connector. =A0From what I can see, it seems that the
group is determined via the MCPermissions.asmx web service and then
each user in that group is iterated over and the SID for those users
are extracted.

Is this the case? =A0If so, are groups created in Sharepoint and AD
groups treated the same way?

Cheers,

Will




--f46d043c7f065ad84204ebbdfd49--
Connection status: Accessing site failed with unexpected Sha= rePoint error code 0x80131600: User cannot be found.