lucene-solr-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jonathan Rochkind <rochk...@jhu.edu>
Subject RE: Using Multiple Cores for Multiple Users
Date Wed, 10 Nov 2010 00:34:29 GMT
If storing in a single index (possibly sharded if you need it), you can simply include a solr
field that specifies the user ID of the saved thing. On the client side, in your application,
simply ensure that there is an fq parameter limiting to the current user, if you want to limit
to the current user's stuff.  Relevancy ranking should work just as if you had 'seperate cores',
there is no relevancy issue. 

It IS true that when your index gets very large, commits will start taking longer, which can
be a problem. I don't mean commits will take longer just because there is more stuff to commit
-- the larger the index, the longer an update to a single document will take to commit. 

In general, i suspect that having dozens or hundreds (or thousands!) of cores is not going
to scale well, it is not going to make good use of your cpu/ram/hd resources.   Not really
the intended use case of multiple cores. 

However, you are probably going to run into some issues with the single index approach too.
In general, how to deal with "multi-tenancy" in Solr is an oft-asked question that there doesn't
seem to be any "just works and does everything for you without needing to think about it"
solution for in solr. Judging from past thread. I am not a Solr developer or expert. 

________________________________________
From: Markus Jelsma [markus.jelsma@openindex.io]
Sent: Tuesday, November 09, 2010 6:57 PM
To: solr-user@lucene.apache.org
Cc: Adam Estrada
Subject: Re: Using Multiple Cores for Multiple Users

Hi,

> All,
>
> I have a web application that requires the user to register and then login
> to gain access to the site. Pretty standard stuff...Now I would like to
> know what the best approach would be to implement a "customized" search
> experience for each user. Would this mean creating a separate core per
> user? I think that this is not possible without restarting Solr after each
> core is added to the multi-core xml file, right?

No, you can dynamically manage cores and parts of their configuration.
Sometimes you must reindex after a change, the same is true for reloading
cores. Check the wiki on this one [1].

>
> My use case is this...User A would like to index 5 RSS feeds and User B
> would like to index 5 completely different RSS feeds and he is not
> interested at all in what User A is interested in. This means that they
> would have to be separate index cores, right?

If you view documents within an rss feed as a separate documents, you can
assign an user ID to those documents, creating a multi user index with rss
documents per user, or group or whatever.

Having a core per user isn't a good idea if you have many users.  It takes up
additional memory and disk space, doesn't share caches etc.  There is also
more maintenance and your need some support scripts to dynamically create new
cores - Solr currently doesn't create a new core directory structure.

But, reindexing a very large index takes up a lot more time and resources and
relevancy might be an issue depending on the rss feeds' contents.

>
> What is the best approach for this kind of thing?

I'd usually store the feeds in a single index and shard if it's too many for a
single server with your specifications. Unless the demands are too specific.

>
> Thanks in advance,
> Adam

[1]: http://wiki.apache.org/solr/CoreAdmin

Cheers

Mime
View raw message