lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From DBSight <dbsi...@gmail.com>
Subject Re: Recent Content - Lucene vs. DB SELECT / DB Triggers / Memcached
Date Fri, 04 Mar 2011 23:34:07 GMT
Can you just maintain the site-wide top 5 by combining top 5 of each Shards?
It can be done in memory and just O(1) operations. No Lucene is needed.

-- 

--

Chris Lu

-------------------------

Instant Scalable Full-Text Search On Any Database/Application

site: http://www.dbsight.net

demo: http://search.dbsight.com

Lucene Database Search in 3 minutes: 
http://wiki.dbsight.com/index.php?title=Create_Lucene_Database_Search_in_3_minutes

On 2011/3/4 9:59, BrightMinds Dev wrote:
> We are developing a large 4-tier multi-server app that will accept 
> Questions and related Comments supplied by users.  There will be 
> 100K's of users that live in Shards.  Also, ideally there would be no 
> delay in adding content and seeing it in recent results but to make 
> the system performant a delay is acceptable.
>
> On the main page we will have 2 panels with the 5 most recent 
> site-wide Questions and 5 most recent site-wide Comments.
> On the user's profile page we would display similar panels except they 
> would only consist of links that pertain to that user.
>
>
> There are essentially 4 design choices:
>
> 1) Do periodic DB SELECT calls and cache for site-wide content "AND" 
> do live DB SELECT calls for user specific content.  Not wonderful but ok.
>
> 2) Use DB triggers to manage tables for site-wide content, though as 
> content is sharded we would need to aggregate and resort based on 
> results obtained from all shards "AND" do live DB SELECT calls for 
> user specific content.  Sounds awful... .
>
> 3) Use a solution like memcached however having multiple servers add 
> and prune site-wide content to cache seems like it would be a synchro 
> nightmare "AND" do live DB SELECT calls for user specific content.  
> Nix that... .
>
> 4) Use Lucene to obtain most recent site-wide content (and better yet 
> push it into memcached) "AND" use Lucene to retrieve live user 
> specific content
>
>
> Is this a natural fit for Lucene?
>
> I understood Lucene is very performant but is finding the last X 
> documents based on a long (timestamp) something that it would do very 
> well?
>
> Or is Lucene not appropriate and should we be considering something else?
>
> Personally, I think Lucene is the best choice... but would like to 
> hear the thoughts of others.
>
> Thanks,
>
> --Nikolaos
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
>



Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message