lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Michael McCandless (JIRA)" <j...@apache.org>
Subject [jira] Commented: (LUCENE-2280) IndexWriter.optimize() throws NullPointerException
Date Fri, 19 Mar 2010 17:52:27 GMT

    [ https://issues.apache.org/jira/browse/LUCENE-2280?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12847471#action_12847471
] 

Michael McCandless commented on LUCENE-2280:
--------------------------------------------

bq. 1. To fix this issue if i disable the optimize (remove the call to IndexWriter.optimize()
from my code) will that create any problem in the long run? if yes what kind of problems it
may create?

Most apps never need to optimize.

You should only optimize if your search performance is not good enough, and really, before
using optimize you should explore other ideas (see ImproveSearchingSpeed on the wiki).

bq. 2. Just by knowing the scenario, that after a NullPointerException, index file gets deleted,
can you provide me a patch where any kind of exception is handled by the Lucene API and my
index remains untouched?

Can't make a patch here until we understand what's actually happening...

It looks like your index got truncated at IW 443.  Hmm actually it looks like IW 442 never
successfully closed.  (do a grep for ": at close:" and you'll see it's missing).

Maybe this is the problem.  Because every merge kept hitting exceptions it looks like IW 442,
even on close, was throwing the exceptions, and never closed properly.

Can you try using IW.close(false) in your finally clause?  This aborts all running merges
and closes the index and likely will workaround whatever (seeming like a JRE bug) is causing
the NPE during merging.

Also, you could try using autoCommit=false.


> IndexWriter.optimize() throws NullPointerException
> --------------------------------------------------
>
>                 Key: LUCENE-2280
>                 URL: https://issues.apache.org/jira/browse/LUCENE-2280
>             Project: Lucene - Java
>          Issue Type: Bug
>          Components: Index
>    Affects Versions: 2.3.2
>         Environment: Win 2003, lucene version 2.3.2, IBM JRE 1.6
>            Reporter: Ritesh Nigam
>         Attachments: lucene.jar, lucene.zip
>
>
> I am using lucene 2.3.2 search APIs for my application, i am indexing 45GB database which
creates approax 200MB index file, after finishing the indexing and while running optimize()
i can see NullPointerExcception thrown in my log and index file is getting corrupted, log
says
> ------------------------------------------------------------------------
> Caused by: 
> java.lang.NullPointerException
> 	at org.apache.lucene.store.BufferedIndexOutput.writeBytes(BufferedIndexOutput.java:49)
> 	at org.apache.lucene.store.IndexOutput.writeBytes(IndexOutput.java:40)
> 	at org.apache.lucene.index.SegmentMerger.mergeNorms(SegmentMerger.java:566)
> 	at org.apache.lucene.index.SegmentMerger.merge(SegmentMerger.java:135)
> 	at org.apache.lucene.index.IndexWriter.mergeMiddle(IndexWriter.java:3273)
> 	at org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:2968)
> 	at org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:240)
> ------------------------------------------------------------------------
> and this is happening quite frequently, although I am not able to reproduce it on demand,
I saw an issue logged which is some what related to mine issue (http://mail-archives.apache.org/mod_mbox/lucene-solr-user/200809.mbox/%3C6E4A40DB-5EFC-42DA-A857-D59F4EC3496B@mikemccandless.com%3E)
but the only difference here is I am not using Store.Compress for my fields, i am using Store.NO
instead. please note that I am using IBM JRE for my application.
> Is this an issue with lucene?, if yes it is fixed in which version?

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org


Mime
View raw message