lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Monsur Hossain" <mon...@monsur.com>
Subject RE: indexdir/segments (No such file or directory) lock file present..
Date Thu, 12 May 2005 14:47:18 GMT
Ramya.  I don't have an answer to your specific lock file question, but
a couple thoughts.

You say you're using multiple threads to index 50,000 documents.  Have
you tried a single thread version first?  I'd try that, and then scale
out to multiple threads as needed.  We index over ten times that many
documents, and the entire application finishes in under an hour (this
includes loading the data from a relational database.  However this is
on a fast server, your results may vary).

Also, you're using a try/catch block to dictate programming logic, which
could be causing issues.  Instead, try something like:

boolean create = !DirectoryExists(d);
index = new IndexWriter(d, new StandardAnalyzer(), create);

The DirectoryExists() function just indicates whether your index exists
(you could also check if the segments file exists). 

Monsur




> -----Original Message-----
> From: Ramya [mailto:ramya_krishnan@sifycorp.com] 
> Sent: Thursday, May 12, 2005 10:33 AM
> To: java-user@lucene.apache.org
> Subject: indexdir/segments (No such file or directory) lock 
> file present..
> 
> 
> 1. I am trying to pump in large number of documents( to the tune of 
> 50000) ... I use muliple threads and i depend on the internal 
> locks of 
> lucene to synchronize the write access to the index.
> 
> try
>             {
>                 index = new IndexWriter(d, new 
> StandardAnalyzer(), false);
>             }
>             catch(IOException e)
>             {
>                 index = new IndexWriter("indexdir", new 
> StandardAnalyzer(), true);
>             }
>             index.addDocument(x.doc);
>             index.close();
> 
> <then i add a docu
> 
> I have the following problem.
> 1. Somehow one lock file remains in the tmp directory. When this 
> happens  i get the error
> FileNotFoundException: 
> /root/mails/mailindex/indexdir/segments (No such 
> file or directory)
> 2. I thought when the index is closed the lock is released 
> internally. 
> If it had been the case y would the lock file remain in the tm dir??
> 
> 2. The size of the index file is too large(Almost 50% of the original 
> document). I use the standard analyzer. Will using a 
> different analyzer 
> help. What would i loose if i use a different one.
> 
> 
> 
> I use the version lucene-1.4.3
> 
> 
> Any hellp would be appreciated.
> 
> 
> 
> 
> -- 
> Thanks and regards
> Ramya Krishnan
> 
> ********** DISCLAIMER **********
> Information contained and transmitted by this E-MAIL is 
> proprietary to 
> Sify Limited and is intended for use only by the individual 
> or entity to 
> which it is addressed, and may contain information that is 
> privileged, 
> confidential or exempt from disclosure under applicable law. 
> If this is a 
> forwarded message, the content of this E-MAIL may not have 
> been sent with 
> the authority of the Company. If you are not the intended 
> recipient, an 
> agent of the intended recipient or a  person responsible for 
> delivering the 
> information to the named recipient,  you are notified that any use, 
> distribution, transmission, printing, copying or 
> dissemination of this 
> information in any way or in any manner is strictly 
> prohibited. If you have 
> received this communication in error, please delete this mail 
> & notify us 
> immediately at admin@sifycorp.com
> 
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
> 
> 


---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org


Mime
View raw message