lucene-solr-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Walter Underwood <wun...@wunderwood.org>
Subject Re: Bitwise operation
Date Thu, 21 Mar 2013 15:08:49 GMT
How often is "frequently"? If it is 1000/second, you have a problem, but you'd have a problem
with most solutions.

Measure or estimate how many documents are affected, how often. Then set a latency for how
long you can wait before the change is visible.

With those, you can evaluate solutions. Without those, you'll never know if it works, even
after you build it.

Hundreds of thousands of documents is a small to moderate sized index. At Netflix, we reindexed
a 250,000 doc index in 20 minutes. That was Solr 1.3 -- Solr is much faster now.

I would go ahead with the dynamic boolean fields solution and measure it. If the performance
is close, use SSD for storage or machines with lots of RAM available for file buffers.

Atomic field-level updates may be helpful: http://wiki.apache.org/solr/Atomic_Updates

Even if you need to use a different approach, you'll know Solr a lot better after using the
straightforward design.

wunder

On Mar 21, 2013, at 1:52 AM, Christopher ARZUR wrote:

> @Jan Høydahl : do you mean "cutom filter" ?
> @Walter Underwood : I also agree with you, I'd only use native functions of Solr, but
I do not know how to solve my problem ... My ACLs are composed of thousands of groups (inheritance)
that have deny / allow user rights and who themselves have deny / allow rights and these rights
may change frequently.
> 
> I do not really see any solution to adopt, thank you for your help
> 
> Le 20/03/2013 15:48, Walter Underwood a écrit :
>> I agree. Your first step should not be trying to make Solr work they way your think
it should. Try really hard to use the existing features, they are there because they solve
a LOT of problems.
>> 
>> Updates are pretty fast, really.
>> 
>> wunder
>> 
>> On Mar 20, 2013, at 2:36 AM, Jan Høydahl wrote:
>> 
>>> Don't try to optimize something which is not a problem.
>>> 
>>> This is what "everyone" does - update documents when ACLs for those documents
change, even with multi-million documents. It works like a charm. Or do you have a special
usecase where permissions for an average document changes several times a day? If not, you
should be fine!
>>> 
>>> --
>>> Jan Høydahl, search solution architect
>>> Cominvent AS - www.cominvent.com
>>> Solr Training - www.solrtraining.com
>>> 
>>> 20. mars 2013 kl. 10:01 skrev Christopher ARZUR <christopher.arzur@cognix-systems.com>:
>>> 
>>>> Hello and thank you for your answers.
>>>> I'll try to explain my problem a little better:
>>>> 
>>>> The goal is to manage ACLs via Solr without reindex the documents at each
change of permission. I have hundreds of thousands of documents, users and groups and permissions
(allow / denied) or each of these groups of these users.
>>>> 
>>>> I read a lot of things about this and it seems that the binary compraison
is the best solution ... but I may be wrong :/
>>>> 
>>>> To do this I saw a plugin here: https://issues.apache.org/jira/browse/SOLR-1913
with an example of this: https://docs.google.com/document/d/10HuqHkYjaEm6Q2ZrRCI0QQMLbbqtRv_NXDHcTjfCRfU/edit?
fold = 1
>>>> 
>>>> Unfortunately I have not managed to successfully install the plugin although
I followed the steps it seems ...
>>>> 
>>>> At the moment I left on a solution that is to use dynamic boolean fields
but I'm afraid to face performance issues on important volumetries in terms of documents and
fees.
>>>> 
>>>> I hope I have been more specific, I await your ideas because I'm not having
chosen the right solution ... small clarification: I discovered solr only a few months ago:
s
>>>> 
>>>> Thanks,
>>>> Christopher




Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message