lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Cam Bazz" <>
Subject Re: Realtime Search for Social Networks Collaboration
Date Thu, 04 Sep 2008 13:08:26 GMT
Hello Jason,
I have been trying to do this for a long time on my own. keep up the good

What I tried was a document cache using apache collections. and before a
indexwrite/delete i would sync the cache with index.

I am waiting for lucene 2.4 to proceed. (query by delete)


On Wed, Sep 3, 2008 at 10:20 PM, Jason Rutherglen <> wrote:

> Hello all,
> I don't mean this to sound like a solicitation.  I've been working on
> realtime search and created some Lucene patches etc.  I am wondering
> if there are social networks (or anyone else) out there who would be
> interested in collaborating with Apache on realtime search to get it
> to the point it can be used in production.  It is a challenging
> problem that only Google has solved and made to scale.  I've been
> working on the problem for a while and though a lot has been
> completed, there is still a lot more to do and collaboration amongst
> the most probable users (social networks) seems like a good thing to
> try to do at this point.  I guess I'm saying it seems like a hard
> enough problem that perhaps it's best to work together on it rather
> than each company try to complete their own.  However I could be
> wrong.
> Realtime search benefits social networks by providing a scalable
> searchable alternative to large Mysql implementations.  Mysql I have
> heard is difficult to scale at a certain point.  Apparently Google has
> created things like BigTable (a large database) and an online service
> called GData (which Google has not published any whitepapers on the
> technology underneath) to address scaling large database systems.
> BigTable does not offer search.   GData does and is used by all of
> Google's web services instead of something like Mysql (this is at
> least how I understand it).  Social networks usually grow and so
> scaling is continually an issue.  It is possible to build a realtime
> search system that scales linearly, something that I have heard
> becomes difficult with Mysql.  There is an article that discusses some
> of these issues
>  I
> don't think the current GData implementation is perfect and there is a
> lot that can be improved on.  It might be helpful to figure out
> together what helpful things can be added.
> If this sounds like something of interest to anyone feel free to send
> your input.
> Take care,
> Jason
> ---------------------------------------------------------------------
> To unsubscribe, e-mail:
> For additional commands, e-mail:

  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message