lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Shai Erera <ser...@gmail.com>
Subject Re: Lucene powers Twitter's search
Date Thu, 07 Oct 2010 06:48:08 GMT
Oops, forgot to paste the link from which the quote was taken:

http://engineering.twitter.com/2010/10/twitters-new-search-architecture.html

Shai

On Thu, Oct 7, 2010 at 8:46 AM, Shai Erera <serera@gmail.com> wrote:

> I came across this post today:
> http://techcrunch.com/2010/10/06/new-twitter-search
>
> And continued reading these:
> http://techcrunch.com/2010/10/06/twitter-search-lives/
>
> From the latter:
>
> Modified Lucene
>
> Lucene is great, but in its current form it has several shortcomings for
> real-time search. That’s why we rewrote big parts of the
> core in-memory data structures, especially the posting lists, while still
> supporting Lucene’s standard APIs. This allows us to
> use Lucene’s search layer almost unmodified. Some of the highlights of our
> changes include:
>
>    - significantly improved garbage collection performance
>    - lock-free data structures and algorithms
>    - posting lists, that are traversable in reverse order
>    - efficient early query termination
>
> We believe that the architecture behind these changes involves several
> interesting topics that pertain to software engineering in
> general (not only search). We hope to continue to share more on these
> improvements.
>
> And, before you ask, we’re planning on contributing all these changes back
> to Lucene; some of which have already made it into
> Lucene’s trunk and its new realtime branch.
>
>
> And as you can read at the bottom of the last post, Michael B. is behind
> all this :-).
>
> FYI
> Shai
>

Mime
View raw message