lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "wenjie zheng" <wenjie.zh...@gmail.com>
Subject Re: weird behavior of IndexReader.indexExists()
Date Tue, 09 May 2006 14:57:05 GMT
I am a little bit confused here.
Since I didn't change the lock directory or disable locking and it is on the
single server sharing same fs, does it mean I am not supposed to get the
error?
But I did get the error. What should I do?

Wenjie
On 5/9/06, Otis Gospodnetic <otis_gospodnetic@yahoo.com> wrote:
>
> If you don't explicitly change the lock directory and do not disable
> locking, the same directory should be used.  I'm assuming this is all done
> on a single server sharing the same file system.  The locks are stored in
> the system's default temporary directory.  That's typically /tmp under
> UNIX/OSX and C:\WINDOWS\TEMP (I think) on Winblows.
>
> Otis
>
> ----- Original Message ----
> From: wenjie zheng <wenjie.zheng@gmail.com>
> To: java-dev@lucene.apache.org
> Sent: Tuesday, May 9, 2006 9:09:47 AM
> Subject: Re: weird behavior of IndexReader.indexExists()
>
> Thanks,
> I guess my question is how to make sure both instances are using the same
> lock directory.
>
> Wenjie
>
> On 5/9/06, Vanlerberghe, Luc <Luc.Vanlerberghe@bvdep.com> wrote:
> >
> > Make sure both instances are using the same lock directory.
> > The segments file should only be read or written while holding the
> > commit lock.
> >
> > If the lock directories don't match, you'll get more 'strange' errors...
> >
> > In Lucene 1.4.2 some methods did not use the lock, this has been patched
> > a couple of months ago.
> >
> > Luc
> >
> > -----Original Message-----
> > From: Andy Hind [mailto:andy.hind@alfresco.org]
> > Sent: dinsdag 9 mei 2006 13:47
> > To: java-dev@lucene.apache.org
> > Subject: RE: weird behavior of IndexReader.indexExists()
> >
> > Hi
> >
> > I think I have discovered this too.
> > It is on my list of issues to raise ....
> >
> > The index exist test looks for the segment file.
> > When the index is committing, and you are unlucky, this file may not be
> > found as the new segments file replaces the old one. The result is the
> > index appears not to exist.
> >
> > Regards
> >
> > Andy
> >
> > -----Original Message-----
> > From: wenjie zheng [mailto:wenjie.zheng@gmail.com]
> > Sent: 08 May 2006 18:57
> > To: java-dev@lucene.apache.org
> > Subject: Re: weird behavior of IndexReader.indexExists()
> >
> > This happens sometimes when number of docs is over 2000. So it's kinda
> > of
> > random.
> >
> > Wenjie
> >
> > On 5/8/06, wenjie zheng <wenjie.zheng@gmail.com> wrote:
> > >
> > > I created an index with more than 30,000 text files.
> > > I used indexExists() to determine either to create a new index or to
> > add
> > > docs to the existing index.
> > >
> > > But when the num of docs in the index was over 3,000 (sometimes 3,400,
> > > sometimes 3,200), the indexExists function returns false, so I ended
> > up
> > > recreating a new index.
> > >
> > > Here is my code:
> > > if index exists, we will add files to it, otherwise, create a new
> > index.
> > > In either case, an IndexingThread will be spawn to do that.
> > >          if(IndexReader.indexExists(indexDir)){
> > >                 logger.info("Working on existing index ...");
> > >                 IndexingThread.startIndexingThread(Username, new
> > > File(propsFile), new File(indexDir), docs,
> > >                               new StandardAnalyzer(), false);
> > >           }else{
> > >                 logger.info("Create a new index ...");
> > >                 IndexingThread.startIndexingThread(Username, new
> > > File(propsFile), new File(indexDir), docs,
> > >                                new StandardAnalyzer(), true);
> > >            }
> > >
> > >
> > > inside the startIndexingThread function, I am calling the following
> > > function to add files to the index:
> > >     /**
> > >      * Add an array of Files to an index
> > >      *
> > >      * @param propsFile the properties file
> > >      * @param indexDir  the folder where index files will be created
> > in
> > >      * @param docs      an array of Files to be add to the index
> > >      * @param analyzer  any Analyzer object
> > >      */
> > >     public void addFiles(File propsFile, File indexDir, File[] docs,
> > > Analyzer analyzer, boolean overwrite) throws Exception {
> > >         Properties props = new Properties(new
> > FileInputStream(propsFile));
> > >
> > >         if(overwrite || IndexReader.indexExists(indexDir)){ //either
> > > overwrite or working on an existing index
> > >             Directory index = FSDirectory.getDirectory(indexDir,
> > > overwrite);
> > >             IndexWriter writer = new IndexWriter(index, analyzer,
> > > overwrite);
> > >
> > >             FileIndexer indexer = new FileIndexer(props);
> > >
> > >             long start = new Date().getTime();
> > >             indexer.index(writer, docs);
> > >             writer.optimize();
> > >             writer.close ();     // close the writer
> > >             index.close();      // close the index Directory
> > >             long end = new Date().getTime();
> > >
> > >             logger.info("Total time: " + (end - start) + " ms");
> > >
> > >         }else{
> > >             logger.error("Index files are not found: " +
> > > indexDir.getAbsolutePath() + ", overwrite = false");
> > >         }
> > >     }
> > >
> > > Thanks,
> > > Wenjie
> > >
> > >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
> > For additional commands, e-mail: java-dev-help@lucene.apache.org
> >
> >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
> > For additional commands, e-mail: java-dev-help@lucene.apache.org
> >
> >
>
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-dev-help@lucene.apache.org
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message