lucene-solr-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Matt Mitchell <goodie...@gmail.com>
Subject Re: api key filtering
Date Sun, 23 Jan 2011 04:43:46 GMT
I think that indexing the access information is going to work nicely, and I
agree that sticking with the simplest/solr way is best. The constraint is
super simple... you can view this set of documents or you can't... based on
an api key: fq=api_key:xxx

Thanks for the feedback on this guys!
Matt

2011/1/22 Jonathan Rochkind <rochkind@jhu.edu>

> If you COULD solve your problem by indexing 'public', or other tokens from
> a limited vocabulary of document roles, in a field -- then I'd definitely
> suggest you look into doing that, rather than doing odd things with Solr
> instead. If the only barrier is not currently having sufficient logic at the
> indexing stage to do that, then it is going to end up being a lot less of a
> headache in the long term to simply add a layer at the indexing stage to add
> that in, then trying to get Solr to do things outside of it's, well,
> 'comfort zone'.
>
> Of course, depending on your requirements, it might not be possible to do
> that, maybe you can't express the semantics in terms of a limited set of
> roles applied to documents. And then maybe your best option really is
> sending an up to 2k element list (not exactly the same list every time,
> presumably) of acceptable documents to Solr with every query, and maybe you
> can get that to work reasonably.  Depending on how many different complete
> lists of documents you have, maybe there's a way to use Solr caches
> effectively in that situation, or maybe that's not even neccesary since
> lookup by unique id should be pretty quick anyway, not really sure.
>
> But if the semantics are possible, much better to work with Solr rather
> than against it, it's going to take a lot less tinkering to get Solr to
> perform well if you can just send an fq=role:public or something, instead of
> a list of document IDs.  You won't need to worry about it, it'll just work,
> because you know you're having Solr do what it's built to do. Totally worth
> a bit of work to add a logic layer at the indexing stage. IMO.
> ________________________________________
> From: Erick Erickson [erickerickson@gmail.com]
> Sent: Saturday, January 22, 2011 4:50 PM
> To: solr-user@lucene.apache.org
> Subject: Re: api key filtering
>
> 1024 is the default number, it can be increased. See MaxBooleanClauses
> in solrconfig.xml
>
> This shouldn't be a problem with 2K clauses, but expanding it to tens of
> thousands is probably a mistake (but test to be sure).
>
> Best
> Erick
>
> On Sat, Jan 22, 2011 at 3:50 PM, Matt Mitchell <goodieboy@gmail.com>
> wrote:
>
> > Hey thanks I'll definitely have a read. The only problem with this
> though,
> > is that our api is a thin layer of app-code, with solr only (no db), we
> > index data from our sql db into solr, and push the index off for
> > consumption.
> >
> > The only other idea I had was to send a list of the allowed document ids
> > along with every solr query, but then I'm sure I'd run into a filter
> query
> > limit. Each key could be associated with up to 2k documents, so that's 2k
> > values in an fq which would probably be too many for lucene (I think its
> > limit 1024).
> >
> > Matt
> >
> > On Sat, Jan 22, 2011 at 3:40 PM, Dennis Gearon <gearond@sbcglobal.net
> > >wrote:
> >
> > > The only way that you would have that many api keys per record, is if
> one
> > > of
> > > them represented 'public', right? 'public' is a ROLE. Your answer is to
> > use
> > > RBAC
> > > style techniques.
> > >
> > >
> > > Here are some links that I have on the subject. What I'm thinking of
> > doing
> > > is:
> > > Sorry for formatting, Firefox is freaking out. I cut and pasted these
> > from
> > > an
> > > email from my sent box. I hope the links came out.
> > >
> > >
> > > Part 1
> > >
> > >
> > >
> >
> http://www.xaprb.com/blog/2006/08/16/how-to-build-role-based-access-control-in-sql/
> > >
> > >
> > > Part2
> > > Role-based access control in SQL, part 2 at Xaprb
> > >
> > >
> > >
> > >
> > >
> > > ACL/RBAC Bookmarks ALL
> > >
> > > UserRbac - symfony - Trac
> > > A Role-Based Access Control (RBAC) system for PHP
> > > Appendix C: Task-Field Access
> > > Role-based access control in SQL, part 2 at Xaprb
> > > PHP Access Control - PHP5 CMS Framework Development | PHP Zone
> > > Linux file and directory permissions
> > > MySQL :: MySQL 5.0 Reference Manual :: C.5.4.1 How to Reset the Root
> > > Password
> > > per RECORD/Entity permissions? - symfony users | Google Groups
> > > Special Topics: Authentication and Authorization | The Definitive Guide
> > to
> > > Yii |
> > > Yii Framework
> > >
> > > att.net Mail (gearond@sbcglobal.net)
> > > Solr - User - Modelling Access Control
> > > PHP Generic Access Control Lists
> > > Row-level Model Access Control for CakePHP « some flot, some jet
> > > Row-level Model Access Control for CakePHP « some flot, some jet
> > > Yahoo! GeoCities: Get a web site with easy-to-use site building tools.
> > > Class that acts as a client to a JSON service : JSON « GWT « Java
> > > Juozas Kaziukėnas devBlog
> > > Re: [symfony-users] Implementing an existing ACL API in symfony
> > > php - CakePHP ACL Database Setup: ARO / ACO structure? - Stack Overflow
> > > W3C ACL System
> > > makeAclTables.sql
> > > SchemaWeb - Classes And Properties - ACL Schema
> > > Reardon's Ruminations: Spring Security ACL Schema for Oracle
> > > trunk/modules/auth/libraries/Khacl.php | Source/SVN | Assembla
> > > Acl.php - kohana-mptt - Project Hosting on Google Code
> > > Asynchronous JavaScript Technology and XML (Ajax) With the Java
> Platform
> > > The page cannot be found
> > >
> > >
> > >  Dennis Gearon
> > >
> > >
> > > Signature Warning
> > > ----------------
> > > It is always a good idea to learn from your own mistakes. It is usually
> a
> > > better
> > > idea to learn from others’ mistakes, so you do not have to make them
> > > yourself.
> > > from 'http://blogs.techrepublic.com.com/security/?p=4501&tag=nl.e036'
> > >
> > >
> > > EARTH has a Right To Life,
> > > otherwise we all die.
> > >
> > >
> > >
> > > ----- Original Message ----
> > > From: Matt Mitchell <goodieboy@gmail.com>
> > > To: solr-user@lucene.apache.org
> > > Sent: Sat, January 22, 2011 11:48:22 AM
> > > Subject: api key filtering
> > >
> > > Just wanted to see if others are handling this in some special way, but
> I
> > > think this is pretty simple.
> > >
> > > We have a database of api keys that map to "allowed" db records. I'm
> > > planning on indexing the db records into solr, along with their api
> keys
> > in
> > > an indexed, non-stored, multi-valued field. Then, to query for docs
> that
> > > belong to a particular api key, they'll be queried using a filter query
> > on
> > > api_key.
> > >
> > > The only concern of mine is that, what if we end up with 100k api_keys?
> > > Would it be a problem to have 100k non-stored keys in each document? We
> > > have
> > > about 500k documents total.
> > >
> > > Matt
> > >
> > >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message