lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Itamar Syn-Hershko <>
Subject Re: Recent Content - Lucene vs. DB SELECT / DB Triggers / Memcached
Date Wed, 09 Mar 2011 19:17:12 GMT
(sorry for picking this up so late...)

This sounds like a perfect fit for document DBs like CouchDB and MongoDB 
- based on your architecture and data structure.

They are designed for multi-server applications, and use Map/Reduce 
which will give you Lucene operations directly from your DB, no DB-index 
sync necessary.

On 04/03/2011 19:59, BrightMinds Dev wrote:

> We are developing a large 4-tier multi-server app that will accept
> Questions and related Comments supplied by users.  There will be
> 100K's of users that live in Shards.  Also, ideally there would be no
> delay in adding content and seeing it in recent results but to make
> the system performant a delay is acceptable.
> On the main page we will have 2 panels with the 5 most recent
> site-wide Questions and 5 most recent site-wide Comments.
> On the user's profile page we would display similar panels except they
> would only consist of links that pertain to that user.
> There are essentially 4 design choices:
> 1) Do periodic DB SELECT calls and cache for site-wide content "AND"
> do live DB SELECT calls for user specific content.  Not wonderful but ok.
> 2) Use DB triggers to manage tables for site-wide content, though as
> content is sharded we would need to aggregate and resort based on
> results obtained from all shards "AND" do live DB SELECT calls for
> user specific content.  Sounds awful... .
> 3) Use a solution like memcached however having multiple servers add
> and prune site-wide content to cache seems like it would be a synchro
> nightmare "AND" do live DB SELECT calls for user specific content.
> Nix that... .
> 4) Use Lucene to obtain most recent site-wide content (and better yet
> push it into memcached) "AND" use Lucene to retrieve live user
> specific content
> Is this a natural fit for Lucene?
> I understood Lucene is very performant but is finding the last X
> documents based on a long (timestamp) something that it would do very
> well?
> Or is Lucene not appropriate and should we be considering something else?
> Personally, I think Lucene is the best choice... but would like to
> hear the thoughts of others.
> Thanks,
> --Nikolaos
> ---------------------------------------------------------------------
> To unsubscribe, e-mail:
> For additional commands, e-mail:

To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message