lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "wenjie zheng" <wenjie.zh...@gmail.com>
Subject Re: weird behavior of IndexReader.indexExists()
Date Tue, 09 May 2006 13:09:47 GMT
Thanks,
I guess my question is how to make sure both instances are using the same
lock directory.

Wenjie

On 5/9/06, Vanlerberghe, Luc <Luc.Vanlerberghe@bvdep.com> wrote:
>
> Make sure both instances are using the same lock directory.
> The segments file should only be read or written while holding the
> commit lock.
>
> If the lock directories don't match, you'll get more 'strange' errors...
>
> In Lucene 1.4.2 some methods did not use the lock, this has been patched
> a couple of months ago.
>
> Luc
>
> -----Original Message-----
> From: Andy Hind [mailto:andy.hind@alfresco.org]
> Sent: dinsdag 9 mei 2006 13:47
> To: java-dev@lucene.apache.org
> Subject: RE: weird behavior of IndexReader.indexExists()
>
> Hi
>
> I think I have discovered this too.
> It is on my list of issues to raise ....
>
> The index exist test looks for the segment file.
> When the index is committing, and you are unlucky, this file may not be
> found as the new segments file replaces the old one. The result is the
> index appears not to exist.
>
> Regards
>
> Andy
>
> -----Original Message-----
> From: wenjie zheng [mailto:wenjie.zheng@gmail.com]
> Sent: 08 May 2006 18:57
> To: java-dev@lucene.apache.org
> Subject: Re: weird behavior of IndexReader.indexExists()
>
> This happens sometimes when number of docs is over 2000. So it's kinda
> of
> random.
>
> Wenjie
>
> On 5/8/06, wenjie zheng <wenjie.zheng@gmail.com> wrote:
> >
> > I created an index with more than 30,000 text files.
> > I used indexExists() to determine either to create a new index or to
> add
> > docs to the existing index.
> >
> > But when the num of docs in the index was over 3,000 (sometimes 3,400,
> > sometimes 3,200), the indexExists function returns false, so I ended
> up
> > recreating a new index.
> >
> > Here is my code:
> > if index exists, we will add files to it, otherwise, create a new
> index.
> > In either case, an IndexingThread will be spawn to do that.
> >          if(IndexReader.indexExists(indexDir)){
> >                 logger.info("Working on existing index ...");
> >                 IndexingThread.startIndexingThread(Username, new
> > File(propsFile), new File(indexDir), docs,
> >                               new StandardAnalyzer(), false);
> >           }else{
> >                 logger.info("Create a new index ...");
> >                 IndexingThread.startIndexingThread(Username, new
> > File(propsFile), new File(indexDir), docs,
> >                                new StandardAnalyzer(), true);
> >            }
> >
> >
> > inside the startIndexingThread function, I am calling the following
> > function to add files to the index:
> >     /**
> >      * Add an array of Files to an index
> >      *
> >      * @param propsFile the properties file
> >      * @param indexDir  the folder where index files will be created
> in
> >      * @param docs      an array of Files to be add to the index
> >      * @param analyzer  any Analyzer object
> >      */
> >     public void addFiles(File propsFile, File indexDir, File[] docs,
> > Analyzer analyzer, boolean overwrite) throws Exception {
> >         Properties props = new Properties(new
> FileInputStream(propsFile));
> >
> >         if(overwrite || IndexReader.indexExists(indexDir)){ //either
> > overwrite or working on an existing index
> >             Directory index = FSDirectory.getDirectory(indexDir,
> > overwrite);
> >             IndexWriter writer = new IndexWriter(index, analyzer,
> > overwrite);
> >
> >             FileIndexer indexer = new FileIndexer(props);
> >
> >             long start = new Date().getTime();
> >             indexer.index(writer, docs);
> >             writer.optimize();
> >             writer.close ();     // close the writer
> >             index.close();      // close the index Directory
> >             long end = new Date().getTime();
> >
> >             logger.info("Total time: " + (end - start) + " ms");
> >
> >         }else{
> >             logger.error("Index files are not found: " +
> > indexDir.getAbsolutePath() + ", overwrite = false");
> >         }
> >     }
> >
> > Thanks,
> > Wenjie
> >
> >
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-dev-help@lucene.apache.org
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-dev-help@lucene.apache.org
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message