lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Shai Erera <ser...@gmail.com>
Subject Lucene powers Twitter's search
Date Thu, 07 Oct 2010 06:46:26 GMT
I came across this post today:
http://techcrunch.com/2010/10/06/new-twitter-search

And continued reading these:
http://techcrunch.com/2010/10/06/twitter-search-lives/

>From the latter:

Modified Lucene

Lucene is great, but in its current form it has several shortcomings for
real-time search. That’s why we rewrote big parts of the
core in-memory data structures, especially the posting lists, while still
supporting Lucene’s standard APIs. This allows us to
use Lucene’s search layer almost unmodified. Some of the highlights of our
changes include:

   - significantly improved garbage collection performance
   - lock-free data structures and algorithms
   - posting lists, that are traversable in reverse order
   - efficient early query termination

We believe that the architecture behind these changes involves several
interesting topics that pertain to software engineering in
general (not only search). We hope to continue to share more on these
improvements.

And, before you ask, we’re planning on contributing all these changes back
to Lucene; some of which have already made it into
Lucene’s trunk and its new realtime branch.


And as you can read at the bottom of the last post, Michael B. is behind all
this :-).

FYI
Shai

Mime
View raw message