Return-Path: Delivered-To: apmail-lucene-dev-archive@www.apache.org Received: (qmail 44629 invoked from network); 20 Apr 2010 15:09:38 -0000 Received: from unknown (HELO mail.apache.org) (140.211.11.3) by 140.211.11.9 with SMTP; 20 Apr 2010 15:09:38 -0000 Received: (qmail 59463 invoked by uid 500); 20 Apr 2010 15:09:36 -0000 Delivered-To: apmail-lucene-dev-archive@lucene.apache.org Received: (qmail 59264 invoked by uid 500); 20 Apr 2010 15:09:35 -0000 Mailing-List: contact dev-help@lucene.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@lucene.apache.org Delivered-To: mailing list dev@lucene.apache.org Received: (qmail 59241 invoked by uid 99); 20 Apr 2010 15:09:35 -0000 Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 20 Apr 2010 15:09:35 +0000 X-ASF-Spam-Status: No, hits=-0.1 required=10.0 tests=HTML_MESSAGE,RCVD_IN_DNSWL_MED,SPF_PASS,WEIRD_PORT X-Spam-Check-By: apache.org Received-SPF: pass (nike.apache.org: domain of karl.wright@nokia.com designates 192.100.122.230 as permitted sender) Received: from [192.100.122.230] (HELO mgw-mx03.nokia.com) (192.100.122.230) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 20 Apr 2010 15:09:27 +0000 Received: from esebh105.NOE.Nokia.com (esebh105.ntc.nokia.com [172.21.138.211]) by mgw-mx03.nokia.com (Switch-3.3.3/Switch-3.3.3) with ESMTP id o3KF8TR4024031; Tue, 20 Apr 2010 18:09:04 +0300 Received: from vaebh104.NOE.Nokia.com ([10.160.244.30]) by esebh105.NOE.Nokia.com with Microsoft SMTPSVC(6.0.3790.3959); Tue, 20 Apr 2010 18:08:56 +0300 Received: from smtp.mgd.nokia.com ([65.54.30.8]) by vaebh104.NOE.Nokia.com over TLS secured channel with Microsoft SMTPSVC(6.0.3790.3959); Tue, 20 Apr 2010 18:08:51 +0300 Received: from NOK-EUMSG-01.mgdnok.nokia.com ([65.54.30.86]) by nok-am1mhub-04.mgdnok.nokia.com ([65.54.30.8]) with mapi; Tue, 20 Apr 2010 17:08:50 +0200 From: To: CC: , Date: Tue, 20 Apr 2010 17:08:50 +0200 Subject: RE: FW: Solr and LCF security at query time Thread-Topic: FW: Solr and LCF security at query time Thread-Index: AcrgmAoSjCRo0snSS6Cs69s+m6zVjwAAkPSg Message-ID: References: In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: acceptlanguage: en-US Content-Type: multipart/alternative; boundary="_000_CF3CE3EFBCA3564185DF065952A267C85302A7BC4DNOKEUMSG01mgd_" MIME-Version: 1.0 X-OriginalArrivalTime: 20 Apr 2010 15:08:51.0320 (UTC) FILETIME=[62A7D780:01CAE09B] X-Nokia-AV: Clean X-Virus-Checked: Checked by ClamAV on apache.org --_000_CF3CE3EFBCA3564185DF065952A267C85302A7BC4DNOKEUMSG01mgd_ Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable SOLR-1872 looks exactly like what I was envisioning, from the search query = perspective, although instead of the acl xml file you specify LCF stipulate= s you would dynamically query the lcf-authority-service servlet for the acc= ess tokens themselves. That would get you support for AD, Documentum, Live= Link, Meridio, and Memex for free. It seems likely that this component coul= d be modified to work with LCF with minor effort. The missing component still seems to be AD authentication, which needs a so= lution. Karl ________________________________ From: ext Peter Sturge [mailto:peter.sturge@googlemail.com] Sent: Tuesday, April 20, 2010 10:44 AM To: dev@lucene.apache.org Subject: Re: FW: Solr and LCF security at query time If you want to do this completely within Solr, have a look at: SOLR-1834 and SOLR-1872. These use a SearchComponent plugin for Solr. Thanks, Peter On Tue, Apr 20, 2010 at 1:25 PM, > wrote: FYI ________________________________ From: Wright Karl (Nokia-S/Cambridge) Sent: Tuesday, April 20, 2010 8:16 AM To: 'dominique.bejean@eolya.fr' Cc: 'solr-dev@apache.org'; 'connectors-dev@incu= bator.apache.org'; 'connectors-= user@incubator.apache.org' Subject: RE: Solr and LCF security at query time Dominique, Yes, I am aware of this ticket and contribution. Luckily LCF establishes a= powerful multi-repository security model, even though it doesn't yet do th= e final step of enforcing that model at the search end. LCF allows you to = define multiple authorities to operate against disparate repositories, and = use the appropriate authority to secure any given document. The solr peopl= e are aware of this design, which addresses the issues raised by SOLR-1834 = very nicely. However, as I said before, time is a problem, and the work st= ill needs to be done. I suggest you read up on the actual security model of LCF, and perhaps expe= riment with that and the SOLR-1834 contribution, to see if there is common = ground. One thing we've learned at MetaCarta is that post-filtering for se= curity purposes is expensive, and it is better to modify the queries themse= lves to restrict the results, if possible. I'm not sure which approach SOL= R-1834 takes, although it sounds like it might be the filtering approach. = Still, it would be better than nothing. Please let me know what you find out. Thanks, Karl ________________________________ From: ext Dominique Bejean [mailto:dominique.bejean@eolya.fr] Sent: Tuesday, April 20, 2010 8:03 AM To: Wright Karl (Nokia-S/Cambridge) Cc: connectors-user@incubator.apache.org; connectors-dev@incubator.apache.org Subject: Re: Solr and LCF security at query time Karl, Thank you for your reply. I made some research today and I found this : http://freesurf001.appspot.com/issues.apache.org/jira/browse/SOLR-1834 http://demo.findwise.se:8880/SolrSecurity/ Sorl security model have to be able to filter result list with items coming= from various sources at the same time (livelink, documentum, file system, = ...). Big subject :) Dominique Le 20/04/10 13:34, karl.wright@nokia.com a = =E9crit : Hi Dominique, At the moment, in order to enforce the LCF security model within Lucene/Sol= r, you will need to build this functionality into whatever client you are u= sing to display the Lucene search results. Specifically, you would need to= take the following steps: (1) Have your users access your search client through Apache. (2) Use the Apache module mod_auth_kerb, combined with LCF's mod_authz_anno= tate, to cause authorization HTTP headers to be transmitted to the client w= ebapp. (3) Have your client webapp alter whatever queries it is doing, to add an a= ppropriate query clause for each of the access tokens transmitted in the he= aders. (This is how it is done at MetaCarta.) Alternatively, you may find a way to do this completely with a web applicat= ion under a Java app server such as Tomcat. I have not yet done the resear= ch to find out whether this is a feasible alternative. Effectively, what y= ou need something like mod_auth_kerb to do is to authenticate your user aga= inst Active Directory, or whomever the authenticator ought to be. JAAS may= be helpful here. There are, of course, intentions to fill out the missing pieces more comple= tely and transparently via a Solr search plugin and/or filter. What has be= en lacking is time. If you are in a position to do development in this are= a, we're happy to have any assistance you might provide. Thanks, Karl ________________________________ From: ext Dominique Bejean [mailto:dominique.bejean@eolya.fr] Sent: Tuesday, April 20, 2010 5:06 AM To: connectors-user@incubator.apache.org Subject: Solr and LCF security at query time Hi, I don't see in LCF wiki how Solr and LCF works together at query time in or= der to remove from the result list the items the user is not allowed to acc= ess. In http://cwiki.apache.org/CONNECTORS/lucene-connectors-framework-concepts.= html, I just see these sentences : " Once all these documents and their access tokens are handed to the search= engine, it is the search engine's job to enforce security by excluding ina= ppropriate documents from the search results. For Lucene, this infrastructu= re is expected to be built on top of Lucene's generic metadata abilities, b= ut has not been implemented at this time." I am not sure to understand. Does this mean that for the moment, it is not = possible for Solr to apply security by using an Authority Connector ? Dominique --_000_CF3CE3EFBCA3564185DF065952A267C85302A7BC4DNOKEUMSG01mgd_ Content-Type: text/html; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable
SOLR-1872 looks exactly like what I was=20 envisioning, from the search query perspective, although instead of the acl= xml=20 file you specify LCF stipulates you would dynamically query the=20 lcf-authority-service servlet for the access tokens themselves.  That = would=20 get you support for AD, Documentum, LiveLink, Meridio, and Memex for=20 free. It seems likely that this component could be modified to work wi= th=20 LCF with minor effort.
 
The missing component still seems to be AD=20 authentication, which needs a solution.
 
Karl

From: ext Peter Sturge=20 [mailto:peter.sturge@googlemail.com]
Sent: Tuesday, April 20, 20= 10=20 10:44 AM
To: dev@lucene.apache.org
Subject: Re: FW: Sol= r and=20 LCF security at query time

If you want to do this completely within Solr, have a look=20 at:
SOLR-1834 and SOLR-1872. These use a SearchComponent plugin for=20 Solr.

Thanks,
Peter



On Tue, Apr 20, 2010 at 1:25 PM, &= lt;karl.wright@nokia.com>= =20 wrote:
FYI

From: Wright Karl (Nokia-S/Cambridge)= =20
Sent: Tuesday, April 20, 2010 8:16 AM
To: 'dominique.bejean@eolya.fr'
Cc: 'solr-dev@apache.org'; 'connectors-dev@incubator.apache.org'; 'connectors-user@incubator.apache.org'
Subject:<= /B> RE:=20 Solr and LCF security at query time

Dominique,
 
Yes, I am=20 aware of this ticket and contribution.  Luckily LCF establishes a=20 powerful multi-repository security model, even though it doesn't yet do t= he=20 final step of enforcing that model at the search end.  LCF allows yo= u to=20 define multiple authorities to operate against disparate repositories, an= d use=20 the appropriate authority to secure any given document.  The solr pe= ople=20 are aware of this design, which addresses the issues raised by SOLR-1834 = very=20 nicely.  However, as I said before, time is a problem, and the work = still=20 needs to be done.
 
I suggest=20 you read up on the actual security model of LCF, and perhaps experiment w= ith=20 that and the SOLR-1834 contribution, to see if there is common ground.&nb= sp;=20 One thing we've learned at MetaCarta is that post-filtering for secu= rity=20 purposes is expensive, and it is better to modify the queries=20 themselves to restrict the results, if possible.  I'm not sure which= =20 approach SOLR-1834 takes, although it sounds like it might be the filteri= ng=20 approach.  Still, it would be better than nothing.
 
Please let=20 me know what you find out.
 
Thanks,
Karl

From: ext Dominique Bejean [mailto:dominique.bejean@eolya.fr]
Sent: Tuesday, = April=20 20, 2010 8:03 AM
To: Wright Karl (Nokia-S/Cambridge)
Cc:<= /B>=20 connectors-user@incubator.apache.org; connectors-dev@incubator.apache.org
Subject: Re:=20 Solr and LCF security at query time

Karl,

Thank you for your reply.

I made some rese= arch=20 today and I found this :
http://freesurf001.appspot.com/issues.apache.org/jira/bro= wse/SOLR-1834
http://demo.findwise.se:8880/SolrSecurity/

Sor= l=20 security model have to be able to filter result list with items coming fr= om=20 various sources at the same time (livelink, documentum, file system, ...)= . Big=20 subject :)

Dominique


Le 20/04/10 13:34, karl.wright@nokia.c= om a=20 =E9crit :=20
Hi=20 Dominique,
 
At the=20 moment, in order to enforce the LCF security model within Lucene/Solr, = you=20 will need to build this functionality into whatever client you are= =20 using to display the Lucene search results.  Specifically, you wou= ld=20 need to take the following steps:
 
(1) Have=20 your users access your search client through Apache.
(2) Use=20 the Apache module mod_auth_kerb, combined with LCF's mod_authz_annotate= , to=20 cause authorization HTTP headers to be transmitted to the client=20 webapp.
(3) Have=20 your client webapp alter whatever queries it is doing, to add an approp= riate=20 query clause for each of the access tokens transmitted in the=20 headers.
 
(This is=20 how it is done at MetaCarta.)
 
Alternatively, you may find a way to do this completely = with a=20 web application under a Java app server such as Tomcat.  I have no= t yet=20 done the research to find out whether this is a feasible alternative.&n= bsp;=20 Effectively, what you need something like mod_auth_kerb to do is to=20 authenticate your user against Active Directory, or whomever the=20 authenticator ought to be.  JAAS may be helpful=20 here.
 
There=20 are, of course, intentions to fill out the missing pieces more complete= ly=20 and transparently via a Solr search plugin and/or filter.  What ha= s=20 been lacking is time.  If you are in a position to do development = in=20 this area, we're happy to have any assistance you might provide. = =20
 
Thanks,
Karl
From: ext Dominique Bejean [mailto:dominique.bejean@eolya.fr]
Sent:= =20 Tuesday, April 20, 2010 5:06 AM
To: connectors-user@incubator.apache.org
Subject:= =20 Solr and LCF security at query time

Hi,

I don't see in LCF wiki how Solr and LCF works togethe= r at=20 query time in order to remove from the result list the items the user i= s not=20 allowed to access.

In http://cwiki.apache.org/CONNECTORS/lucene-connectors-fr= amework-concepts.html,=20 I just see these sentences :

" Once all these documents and the= ir=20 access tokens are handed to the search engine, it is the search engine'= s job=20 to enforce security by excluding inappropriate documents from the searc= h=20 results. For Lucene, = this=20 infrastructure is expected to be built on top of Lucene's generic metad= ata=20 abilities, but has not been implemented at this time."

I am not = sure=20 to understand. Does this mean that for the moment, it is not possible f= or=20 Solr to apply security by using an Authority Connector ?
  
Dominique
=

--_000_CF3CE3EFBCA3564185DF065952A267C85302A7BC4DNOKEUMSG01mgd_--