lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Michael McCandless <luc...@mikemccandless.com>
Subject Re: OutOfMemoryError using IndexWriter
Date Thu, 25 Jun 2009 11:25:44 GMT
OK it looks like no merging was done.

I think the next step is to call
IndexWriter.setMaxBufferedDeleteTerms(1000) and see if that prevents
the OOM.

Mike

On Thu, Jun 25, 2009 at 7:16 AM, stefan<stefan@intermediate.de> wrote:
> Hi,
>
> Here are the result of CheckIndex. I ran this just after I got the OOError.
>
> OK [4 fields]
>    test: terms, freq, prox...OK [509534 terms; 9126904 terms/docs pairs; 4933036 tokens]
>    test: stored fields.......OK [148124 total field count; avg 2 fields per doc]
>    test: term vectors........OK [0 total vector count; avg 0 term/freq vector fields
per doc]
>
>  2 of 7: name=_b docCount=17724
>    compound=true
>    hasProx=true
>    numFiles=2
>    size (MB)=4,514
>    docStoreOffset=0
>    docStoreSegment=_b
>    docStoreIsCompoundFile=true
>    no deletions
>    test: open reader.........OK
>    test: fields, norms.......OK [4 fields]
>    test: terms, freq, prox...OK [122054 terms; 1022477 terms/docs pairs; 1560703 tokens]
>    test: stored fields.......OK [35448 total field count; avg 2 fields per doc]
>    test: term vectors........OK [0 total vector count; avg 0 term/freq vector fields
per doc]
>
>  3 of 7: name=_c docCount=15952
>    compound=true
>    hasProx=true
>    numFiles=2
>    size (MB)=4,539
>    docStoreOffset=17724
>    docStoreSegment=_b
>    docStoreIsCompoundFile=true
>    no deletions
>    test: open reader.........OK
>    test: fields, norms.......OK [4 fields]
>    test: terms, freq, prox...OK [125512 terms; 1047363 terms/docs pairs; 1535701 tokens]
>    test: stored fields.......OK [31904 total field count; avg 2 fields per doc]
>    test: term vectors........OK [0 total vector count; avg 0 term/freq vector fields
per doc]
>
>  4 of 7: name=_d docCount=19975
>    compound=true
>    hasProx=true
>    numFiles=2
>    size (MB)=5,547
>    docStoreOffset=33676
>    docStoreSegment=_b
>    docStoreIsCompoundFile=true
>    no deletions
>    test: open reader.........OK
>    test: fields, norms.......OK [4 fields]
>    test: terms, freq, prox...OK [101563 terms; 1327972 terms/docs pairs; 2390213 tokens]
>    test: stored fields.......OK [39950 total field count; avg 2 fields per doc]
>    test: term vectors........OK [0 total vector count; avg 0 term/freq vector fields
per doc]
>
>  5 of 7: name=_e docCount=24740
>    compound=true
>    hasProx=true
>    numFiles=2
>    size (MB)=5,458
>    docStoreOffset=53651
>    docStoreSegment=_b
>    docStoreIsCompoundFile=true
>    no deletions
>    test: open reader.........OK
>    test: fields, norms.......OK [4 fields]
>    test: terms, freq, prox...OK [94791 terms; 1290085 terms/docs pairs; 2501794 tokens]
>    test: stored fields.......OK [49480 total field count; avg 2 fields per doc]
>    test: term vectors........OK [0 total vector count; avg 0 term/freq vector fields
per doc]
>
>  6 of 7: name=_f docCount=21584
>    compound=true
>    hasProx=true
>    numFiles=2
>    size (MB)=5,914
>    docStoreOffset=0
>    docStoreSegment=_f
>    docStoreIsCompoundFile=true
>    no deletions
>    test: open reader.........OK
>    test: fields, norms.......OK [4 fields]
>    test: terms, freq, prox...OK [92162 terms; 1267882 terms/docs pairs; 2570682 tokens]
>    test: stored fields.......OK [43168 total field count; avg 2 fields per doc]
>    test: term vectors........OK [0 total vector count; avg 0 term/freq vector fields
per doc]
>
>  7 of 7: name=_g docCount=13600
>    compound=true
>    hasProx=true
>    numFiles=2
>    size (MB)=1,664
>    docStoreOffset=21584
>    docStoreSegment=_f
>    docStoreIsCompoundFile=true
>    no deletions
>    test: open reader.........OK
>    test: fields, norms.......OK [4 fields]
>    test: terms, freq, prox...OK [42087 terms; 326152 terms/docs pairs; 667302 tokens]
>    test: stored fields.......OK [27200 total field count; avg 2 fields per doc]
>    test: term vectors........OK [0 total vector count; avg 0 term/freq vector fields
per doc]
>
> No problems were detected with this index.
>
>
> Stefan
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
>

---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org


Mime
View raw message