lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "John Wang" <john.w...@gmail.com>
Subject Re: Does Lucene Supports Billions of data
Date Thu, 01 May 2008 20:10:39 GMT
I am not sure why this is the case, docid is internal to the sub index. As
long as the sub index size is below 2 bil, there is no need for docid to be
long. With multiple indexes, I was thinking having an aggregater which
merges maybe only a page of search result.

Example:

sub index 1: 1 billion docs
sub index 2: 1 billion docs
sub index 3: 1 billion docs

federating search to these subindexes, you represent an index of 3 billion
docs, and all internal doc ids are of type int.

Maybe I am not understanding something.

-John

On Wed, Apr 30, 2008 at 4:10 PM, Daniel Noll <daniel@nuix.com> wrote:

> On Thursday 01 May 2008 00:01:48 John Wang wrote:
> > I am not sure how well lucene would perform with > 2 Billion docs in a
> > single index anyway.
>
> Even if they're in multiple indexes, the doc IDs being ints will still
> prevent
> it going past 2Gi unless you wrap your own framework around it.
>
> Daniel
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message