lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Erick Erickson <erickerick...@gmail.com>
Subject Re: Slow ness of IndexWriter.close()
Date Mon, 13 Jun 2011 12:23:50 GMT
My first question is "what are you trying to do at a higher level"?
Because asking people to check your code without telling us
what you're trying to accomplish makes it difficult to know what
to look at. You might review:

http://wiki.apache.org/solr/UsingMailingLists

That said, at a guess, your update statement will first delete
all the items in the index where the term matches. Is it possible
that you're deleting/adding the same documents over and
over and over again?

Best
Erick

On Mon, Jun 13, 2011 at 7:14 AM, Yogesh Dabhi <ydabhi@asite.com> wrote:
>
>
> Hi,
>
> I try to add and update  document in index
>
> At the start its take only 1 to 2 second but after 50 to 60 document add
> and update
>
> Its take 40 to 50 second or some time its take more then 1 min
>
>
>
> Is there any  way to improve performance ?
>
>
>
> Please help me
>
>
>
> Please check my code
>
>
>
> if(documents!=null && documents.size()>0){
>
>      IndexWriter writer = null;
>
>      boolean createNewIndexDirectory =
> !IndexReader.indexExists(FSDirectory.open(indexDir));
>
>      int createNewIndexReTries = 0;
>
>      Directory directory = FSDirectory.open(indexDir, null);
>
>      while(writer==null) {
>
>            try{
>
>                  directory.setLockFactory(new
> NativeFSLockFactory(indexDir));
>
>
>
>                  if(analyzer == null){
>
>
>                        analyzer = new
> StandardAnalyzer(IAsiteIndexConstants.LUCENE_VERSION,AsiteIndexUtil.getS
> topWordSet());
>
>                  }
>
>                  writer = new IndexWriter(directory, analyzer,
> createNewIndexDirectory,IndexWriter.MaxFieldLength.UNLIMITED);
>
>                  for(Document document : documents){
>
>
> if(document.get(ICustomFormIndexConstants.INDEX_FIELD_FORM_ID)!=null){
>
>                              formId =
> Integer.parseInt(document.get(ICustomFormIndexConstants.INDEX_FIELD_FORM
> _ID));
>
>                        }
>
>                        String strValue =
> NumericUtils.longToPrefixCoded(Long.parseLong(document.get(ICustomFormIn
> dexConstants.INDEX_FIELD_MSG_ID)));
>
>                        term = new
> Term(ICustomFormIndexConstants.INDEX_FIELD_MSG_ID,strValue);
>
>                        writer.updateDocument(term, document);
>
>                  }
>
>            }catch (LockObtainFailedException ex){
>
>                  CheckIndex.Status status = new
> CheckIndex(directory).checkIndex();
>
>                  if(status!=null && status.numBadSegments>0){
>
>                        log.error("Index found corrupted.
> status.numBadSegments ::"+status.numBadSegments);
>
>                        ex.printStackTrace();
>
>                        throw ex;
>
>                  }
>
>            }finally{
>
>                  try{
>
>                        if(writer!=null){
>
>                              writer.close();
>
>                        }
>
>
>
>                  }catch(IOException ex){
>
>                  }
>
>            }
>
>
>
>      }
>
>
>
>
> Thanks & Regards
>
> Yogesh
>
>

---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org


Mime
View raw message