Return-Path: Delivered-To: apmail-lucene-java-user-archive@www.apache.org Received: (qmail 609 invoked from network); 13 Jan 2011 10:24:18 -0000 Received: from hermes.apache.org (HELO mail.apache.org) (140.211.11.3) by minotaur.apache.org with SMTP; 13 Jan 2011 10:24:18 -0000 Received: (qmail 40367 invoked by uid 500); 13 Jan 2011 10:24:16 -0000 Delivered-To: apmail-lucene-java-user-archive@lucene.apache.org Received: (qmail 39834 invoked by uid 500); 13 Jan 2011 10:24:12 -0000 Mailing-List: contact java-user-help@lucene.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: java-user@lucene.apache.org Delivered-To: mailing list java-user@lucene.apache.org Received: (qmail 39821 invoked by uid 99); 13 Jan 2011 10:24:11 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 13 Jan 2011 10:24:11 +0000 X-ASF-Spam-Status: No, hits=1.5 required=10.0 tests=HTML_MESSAGE,RCVD_IN_DNSWL_LOW,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (athena.apache.org: domain of zhitian.zhang@dianping.com designates 209.85.216.169 as permitted sender) Received: from [209.85.216.169] (HELO mail-qy0-f169.google.com) (209.85.216.169) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 13 Jan 2011 10:24:07 +0000 Received: by qyk7 with SMTP id 7so4771669qyk.14 for ; Thu, 13 Jan 2011 02:23:45 -0800 (PST) MIME-Version: 1.0 Received: by 10.224.45.74 with SMTP id d10mr2052833qaf.71.1294914225144; Thu, 13 Jan 2011 02:23:45 -0800 (PST) Received: by 10.220.184.170 with HTTP; Thu, 13 Jan 2011 02:23:45 -0800 (PST) In-Reply-To: References: Date: Thu, 13 Jan 2011 18:23:45 +0800 Message-ID: Subject: Re: Can not delete index file after close the IndexSearcher From: =?GB2312?B?1cXWvszv?= To: java-user@lucene.apache.org Content-Type: multipart/alternative; boundary=001517503bba6782df0499b7b68d --001517503bba6782df0499b7b68d Content-Type: text/plain; charset=GB2312 Content-Transfer-Encoding: quoted-printable Ian, thanks for your response. Your suggestion worked for me. What does oldSearcher.close() do in my code? why I have to close the searcher and oldSearcher together? In my opinion, oldSearcher held index1 while searcher held index2, they are using different resources, the resources held by them should be released seperately. I have another concern for your solution, searcher is a reference created here for user searching out of this code snippet, if I closed and reopen it here, there may be some service down time because there is no open searcher for using. In my original code, searcher opened all the time, so there is no service down time or little, this is the reason I did not close it every time. Do you have any suggestion to keep an alive searcher and the program can also switch the index smoothly? Thanks, Garry =D4=DA 2011=C4=EA1=D4=C213=C8=D5 =CF=C2=CE=E75:47=A3=ACIan Lea =D0=B4=B5=C0=A3=BA > Try adding > > try { searcher.close(); } catch (Exception e) { } > > before > > searcher =3D new IndexSearcher(dir); > > at the top of the loop. > > At the end of a loop searcher is open, and is not closed before being > reassigned. There is probably a better solution along the lines of > only opening new searcher if need to. > > > -- > Ian. > > 2011/1/13 =D5=C5=D6=BE=CC=EF : > > Hi Yuhan, > > > > dir.close() can not solve the problem. > > > > The reason I have to close the old searcher is my program will replace > the > > old index, the code posted here is just a scenario to simplify my > question. > > > > Thanks, > > Garry > > > > =D4=DA 2011=C4=EA1=D4=C213=C8=D5 =C9=CF=CE=E710:45=A3=ACYuhan Zhang =D0=B4=B5=C0=A3=BA > > > >> Hi Garry, > >> > >> I am guessing the directory needs to be closed before opening a new on= e. > >> > >> dir.close(); > >> dir =3D FSDirectory.open(new File(getIndexPath())); > >> > >> why not to open two IndexSearcher objects in an array of two instead o= f > >> swapping them back and forth? > >> it would be a lot easier. > >> > >> yuhan > >> > >> 2011/1/12 =D5=C5=D6=BE=CC=EF > >> > >> > Hi Mike, > >> > > >> > Sorry to make you confused. "lock" means the file handle is held by > some > >> > other progress, the program can not delete it. There is no exception= , > I > >> can > >> > see file.delete() method returns false. If I delete the cfs file in > the > >> OS > >> > manually, the warning is "File was using by another person or progra= m" > >> > > >> > To simplify my question, I made some more code for testing. you can > run > >> it > >> > for reproducing, after two loops, you will see the message e.g. "Can > not > >> > delete file: D:\index\index2\_0.cfs" > >> > > >> > > >> > Thank you very much > >> > > >> > > >> > public class SearchTest > >> > { > >> > > >> > private static final int MAX_RESULT =3D 10000; > >> > > >> > private String indexPath1 =3D "D:\\index\\index1"; > >> > > >> > private String indexPath2 =3D "D:\\index\\index2"; > >> > > >> > private String backupIndexpath =3D "D:\\index\\index3"; > >> > > >> > private String indexPath =3D indexPath1; > >> > > >> > private Analyzer analyzer =3D new > StandardAnalyzer(Version.LUCENE_30); > >> > > >> > private IndexSearcher searcher; > >> > > >> > public void search() > >> > { > >> > while (true) > >> > { > >> > try > >> > { > >> > String keyword =3D "test"; > >> > String fieldName =3D "searchfield"; > >> > > >> > Directory dir =3D FSDirectory.open(new File(indexPat= h)); > >> > > >> > searcher =3D new IndexSearcher(dir); > >> > > >> > QueryParser queryParse =3D new > >> QueryParser(Version.LUCENE_30, > >> > fieldName, analyzer); > >> > Query query =3D queryParse.parse(keyword); > >> > > >> > TopDocs hits =3D searcher.search(query, MAX_RESULT); > >> > int size =3D 5; > >> > if (hits.scoreDocs.length < size) > >> > { > >> > size =3D hits.scoreDocs.length; > >> > } > >> > for (int i =3D 0; i < size; i++) > >> > { > >> > Document doc =3D searcher.doc(hits.scoreDocs[i].d= oc); > >> > String text =3D doc.get(fieldName); > >> > System.out.println("fieldContent is: " + text); > >> > } > >> > > >> > IndexSearcher oldSearcher =3D searcher; > >> > > >> > File newFile =3D new File(getIndexPath()); > >> > for (File file : newFile.listFiles()) > >> > { > >> > if (!file.delete()) > >> > { > >> > System.out.println("Can not delete file: " + > >> > file.getAbsolutePath()); > >> > } > >> > } > >> > > >> > // Copy index File from another folder to this folder > >> > copyDir(new File(backupIndexpath), newFile); > >> > > >> > Directory newDir =3D FSDirectory.open(newFile); > >> > IndexSearcher newSearcher =3D new IndexSearcher(newDi= r); > >> > searcher =3D newSearcher; > >> > > >> > oldSearcher.close(); > >> > > >> > System.out.println("Closed Searcher: " + > >> > oldSearcher.getIndexReader().directory().toString()); > >> > > >> > System.out.println("input 'Q' to quit testing..."); > >> > BufferedReader br =3D new BufferedReader(new > >> > InputStreamReader(System.in)); > >> > > >> > if (br.readLine().trim().equals("Q")) > >> > { > >> > break; > >> > } > >> > } > >> > catch (CorruptIndexException e) > >> > { > >> > e.printStackTrace(); > >> > } > >> > catch (IOException e) > >> > { > >> > e.printStackTrace(); > >> > } > >> > catch (ParseException e) > >> > { > >> > e.printStackTrace(); > >> > } > >> > } > >> > } > >> > > >> > private String getIndexPath() > >> > { > >> > if (indexPath.equals(indexPath1)) > >> > { > >> > indexPath =3D indexPath2; > >> > } > >> > else > >> > { > >> > indexPath =3D indexPath1; > >> > } > >> > > >> > return indexPath; > >> > } > >> > > >> > public static void copyDir(File sourceLocation, File > targetLocation) > >> > throws IOException > >> > { > >> > String[] children =3D sourceLocation.list(); > >> > for (int i =3D 0; i < children.length; i++) > >> > { > >> > InputStream in =3D null; > >> > OutputStream out =3D null; > >> > try > >> > { > >> > in =3D new FileInputStream(new File(sourceLocation, > >> > children[i])); > >> > out =3D new FileOutputStream(new File(targetLocation, > >> > children[i])); > >> > > >> > byte[] buf =3D new byte[1024]; > >> > int len; > >> > while ((len =3D in.read(buf)) > 0) > >> > { > >> > out.write(buf, 0, len); > >> > } > >> > } > >> > catch (FileNotFoundException e) > >> > { > >> > e.printStackTrace(); > >> > } > >> > catch (IOException ioe) > >> > { > >> > ioe.printStackTrace(); > >> > } > >> > finally > >> > { > >> > try > >> > { > >> > if (in !=3D null) > >> > { > >> > in.close(); > >> > } > >> > if (out !=3D null) > >> > { > >> > out.close(); > >> > } > >> > } > >> > catch (IOException e) > >> > { > >> > e.printStackTrace(); > >> > } > >> > } > >> > } > >> > } > >> > > >> > public static void main(String[] args) > >> > { > >> > SearchTest searchTest =3D new SearchTest(); > >> > searchTest.search(); > >> > } > >> > > >> > } > >> > > >> > =D4=DA 2011=C4=EA1=D4=C212=C8=D5 =CF=C2=CE=E711:53=A3=ACMichael McCa= ndless >=D0=B4=B5=C0=A3=BA > >> > > >> > > Hmmm. > >> > > > >> > > When you say "locked" what actually does that mean? Can you post > the > >> > > exception? > >> > > > >> > > Also, can you whittle down your example even more? EG if calling > >> > > this method twice causes the problem, make a method that calls it > >> > > twice and hits the exception and then start simplifying from > there... > >> > > > >> > > Mike > >> > > > >> > > 2011/1/12 =D5=C5=D6=BE=CC=EF : > >> > > > Mike, thanks for your feedback. > >> > > > > >> > > > I verified this in the debug mode, so I just check the folder I > >> closed > >> > in > >> > > > the last loop. Actually, both two folders are locked. > >> > > > > >> > > > tried with new FSDirectory every loop, no help. > >> > > > > >> > > > Garry > >> > > > > >> > > > 2011/1/12 Michael McCandless > >> > > > > >> > > >> When you break out of the loop (user enters 'Q') you don't clos= e > the > >> > > >> current searcher. Could that be it? > >> > > >> > >> > > >> Also you are calling FSDir.open each time but should only do it > once > >> > > >> (though this should be "harmless"). > >> > > >> > >> > > >> Mike > >> > > >> > >> > > >> On Wed, Jan 12, 2011 at 5:39 AM, =D5=C5=D6=BE=CC=EF > > >> > > wrote: > >> > > >> > Dear Luceners, > >> > > >> > > >> > > >> > I'm using lucene-3.0.2 in our app. There is some testing code > for > >> > > >> switching > >> > > >> > index, however, when my code run a couple of times, I found t= he > >> > index > >> > > >> file > >> > > >> > was locked, I can not delete the old index files. > >> > > >> > > >> > > >> > > >> > > >> > The code looks like: > >> > > >> > > >> > > >> > public class SearchTest > >> > > >> > { > >> > > >> > > >> > > >> > private static final int MAX_RESULT =3D 10000; > >> > > >> > > >> > > >> > private String indexPath1 =3D "D:\\index\\index1"; > >> > > >> > private String indexPath2 =3D "D:\\index\\index2"; > >> > > >> > > >> > > >> > private String indexPath =3D indexPath1; > >> > > >> > > >> > > >> > private Analyzer analyzer =3D new > >> > > StandardAnalyzer(Version.LUCENE_30); > >> > > >> > > >> > > >> > private Directory dir =3D null; > >> > > >> > > >> > > >> > private IndexSearcher searcher; > >> > > >> > > >> > > >> > public void search() > >> > > >> > { > >> > > >> > while(true) > >> > > >> > { > >> > > >> > try > >> > > >> > { > >> > > >> > String keyword =3D "test"; > >> > > >> > String fieldName =3D "searchfield"; > >> > > >> > > >> > > >> > if(dir =3D=3D null) > >> > > >> > { > >> > > >> > dir =3D FSDirectory.open(new File(indexPat= h)); > >> > > >> > } > >> > > >> > searcher =3D new IndexSearcher(dir); > >> > > >> > > >> > > >> > QueryParser queryParse =3D new > >> > > >> QueryParser(Version.LUCENE_30, > >> > > >> > fieldName, analyzer); > >> > > >> > Query query =3D queryParse.parse(keyword); > >> > > >> > > >> > > >> > TopDocs hits =3D searcher.search(query, > MAX_RESULT); > >> > > >> > int size =3D 5; > >> > > >> > if(hits.scoreDocs.length < size) > >> > > >> > { > >> > > >> > size =3D hits.scoreDocs.length; > >> > > >> > } > >> > > >> > for (int i =3D 0; i < size; i++) > >> > > >> > { > >> > > >> > Document doc =3D > >> > searcher.doc(hits.scoreDocs[i].doc); > >> > > >> > String text =3D doc.get(fieldName); > >> > > >> > System.out.println("fieldContent is: " + > text); > >> > > >> > } > >> > > >> > > >> > > >> > IndexSearcher oldSearcher =3D searcher; > >> > > >> > dir =3D FSDirectory.open(new > File(getIndexPath())); > >> > > >> > IndexSearcher newSearcher =3D new > IndexSearcher(dir); > >> > > >> > searcher =3D newSearcher; > >> > > >> > > >> > > >> > oldSearcher.close(); > >> > > >> > System.out.println("Closed Searcher: " + > >> > > >> > oldSearcher.getIndexReader().directory().toString()); > >> > > >> > > >> > > >> > System.out.println("input 'Q' to quit > testing..."); > >> > > >> > BufferedReader br =3D new BufferedReader(new > >> > > >> > InputStreamReader(System.in)); > >> > > >> > > >> > > >> > if(br.readLine().trim().equals("Q")) > >> > > >> > { > >> > > >> > break; > >> > > >> > } > >> > > >> > } > >> > > >> > catch (CorruptIndexException e) > >> > > >> > { > >> > > >> > e.printStackTrace(); > >> > > >> > } > >> > > >> > catch (IOException e) > >> > > >> > { > >> > > >> > e.printStackTrace(); > >> > > >> > } > >> > > >> > catch (ParseException e) > >> > > >> > { > >> > > >> > e.printStackTrace(); > >> > > >> > } > >> > > >> > } > >> > > >> > } > >> > > >> > > >> > > >> > private String getIndexPath() > >> > > >> > { > >> > > >> > if(indexPath.equals(indexPath1)) > >> > > >> > { > >> > > >> > indexPath =3D indexPath2; > >> > > >> > } > >> > > >> > else > >> > > >> > { > >> > > >> > indexPath =3D indexPath1; > >> > > >> > } > >> > > >> > > >> > > >> > return indexPath; > >> > > >> > } > >> > > >> > > >> > > >> > public static void main(String[] args) > >> > > >> > { > >> > > >> > SearchTest searchTest =3D new SearchTest(); > >> > > >> > searchTest.search(); > >> > > >> > } > >> > > >> > > >> > > >> > } > >> > > >> > > >> > > >> > Can anybody take a look at the above code snippet? > >> > > >> > > >> > > >> > I want to search on the different index file every time so I > >> created > >> > > two > >> > > >> > different folders and switch them time to time. The index fil= es > in > >> > the > >> > > >> > index1/index2 maybe replaced before the search request comes. > >> > > >> > > >> > > >> > The problem I found is after I ran the above code 2 or more > loops, > >> I > >> > > can > >> > > >> not > >> > > >> > modify/delete the cfs/cfx file in the file system(Windows > 2003), > >> > > although > >> > > >> I > >> > > >> > closed the searcher every time in the code. It seems that the > >> index > >> > > file > >> > > >> is > >> > > >> > not released. > >> > > >> > > >> > > >> > Is the problem caused by the shared reference of searcher? or > some > >> > > shared > >> > > >> > thread in the lucene? > >> > > >> > > >> > > >> > Thanks in advance! > >> > > >> > Garry > >> > > >> > > >> > > >> > >> > > > > >> > > > > >> > > > > >> > > > -- > >> > > > =D5=C5=D6=BE=CC=EF > >> > > > > >> > > > =B4=F3=D6=DA=B5=E3=C6=C0=CD=F8 - =BC=BC=CA=F5=B2=BF > >> > > > =B5=E7=BB=B0=A3=BA52521070 - 1675 > >> > > > > >> > > > >> > > > --------------------------------------------------------------------- > >> > > To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org > >> > > For additional commands, e-mail: java-user-help@lucene.apache.org > >> > > > >> > > > >> > > >> > > >> > -- > >> > =D5=C5=D6=BE=CC=EF > >> > > >> > =B4=F3=D6=DA=B5=E3=C6=C0=CD=F8 - =BC=BC=CA=F5=B2=BF > >> > =B5=E7=BB=B0=A3=BA52521070 - 1675 > >> > > >> > > > > > > > > -- > > =D5=C5=D6=BE=CC=EF > > > > =B4=F3=D6=DA=B5=E3=C6=C0=CD=F8 - =BC=BC=CA=F5=B2=BF > > =B5=E7=BB=B0=A3=BA52521070 - 1675 > > > > --------------------------------------------------------------------- > To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org > For additional commands, e-mail: java-user-help@lucene.apache.org > > --001517503bba6782df0499b7b68d--