lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Michael McCandless <luc...@mikemccandless.com>
Subject Re: indexing 15 million documents to lucene
Date Wed, 06 Jul 2016 22:12:35 GMT
Use threads, only commit at the end (and use a near-real-time reader if you
want to search at points-in-time), increase IW's indexing buffer.

Mike McCandless

http://blog.mikemccandless.com

On Wed, Jul 6, 2016 at 4:37 PM, Nomar Morado <nomar.morado@gmail.com> wrote:

> Hi
>
> I am trying to write 15 million documents (and maybe more) to lucene for
> indexing.
>
> I would try to call writer.commit at some #/byte size of documents.
>
> The entire generated lucene files are about 1 GB total.
>
> My timing is about ~ 15-20 mins.
>
> I don't know if there are other configurations i can leverage to bring down
> the timing down.
>
>
> Any thoughts on this?
>
>
> Thanks,
> Nomar
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message