lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ridwan Habbal <ridwanhab...@hotmail.com>
Subject RE: IndexReader deletes more that expected
Date Thu, 02 Aug 2007 07:56:06 GMT








 




> Subject: RE: IndexReader deletes more that expected> Date: Wed, 1 Aug 2007 09:07:32
-0700> From: steven_parkes@esseff.org> To: java-user@lucene.apache.org> > If I'm
reading this correctly, there's something a little wonky here. In> your example code, you
close the IndexWriter and then, without creating> a new IndexWriter, you call addDocument
again. This shouldn't be> possible (what version of Lucene are you using?)
Yes, you are correct, I close indexWriter and then add more docs. What's wrong? it worked
out fine, and add docs i add will appear to NEW INSTANCES OF INDEX SEARCHERS after calling
close on the indexWriter. 
   As for creating new IndexWriter, I tried to, however i suffered of the lock exception i
got, although i was closing the IndexWriter instance, before creating a new IndexWriter instance!
I don't know why! furthermore, this is useless, for multiThreaded app, cause you can't know
who is still writing to your index, and who has colsed his IndexWriter. Even checking that
the index is locked b4, leads to unnecessary overhead which can be avoided since it works
for me and i can write with one single instance of IndexWriter. >
> > Assuming for the time being that you are creating the IndexWriter again,> the
other issue here is that you shouldn't be able to have a reader and> a writer changing
an index at the same time. There should be a lock> failure. This should occur either in
the Index 
Well, I think that i don't get the problems you expect cause i use the Lucene version that
is shiped by compass distribution (www.compassframework.org) In short, compass to Lucene is
the same of ORM server like hibernate for DBMS like oracle. It's really works fine but i couldn't
understand why compass hides the deleteDocuments(Term) method on IndexWriter class
http://lucene.zones.apache.org:8080/hudson/job/Lucene-Nightly/javadoc/org/apache/lucene/index/IndexWriter.html#deleteDocuments(org.apache.lucene.index.Term)
. This is why i used delete on a reader rather than using it on the same writer instance,
and the only one i have. I couldn't manage my index in one particular situation using compass,
because i had to store data not in the usual way we do (every row in table is a record). So
i think i have to ask The compass team about that. Anyways, if you have coments, you or the
others please do. >
> > Might you be creating your IndexWriters (which you don't show) with the> create
flag always set to true? That will wipe your index each time,> ignoring the locks and cause
all sorts of weird results.
No, I don't create a new Instance of indexWriter, the only one I create is in the Service
constructor, so i create a new clean (has no docs) index only when program starts up. public
LuceneServiceSHImp(String indexDirectory) throws IOException{this.indexDirectory = indexDirectory;standardAnalyzer
= new StandardAnalyzer();indexWriter = new IndexWriter(new java.io.File(indexDirectory), standardAnalyzer,
true);indexWriter.close();}
> > -----Original Message-----> From: Ridwan Habbal [mailto:ridwanhabbal@hotmail.com]
> Sent: Wednesday, August 01, 2007 8:48 AM> To: java-user@lucene.apache.org> Subject:
IndexReader deletes more that expected> > Hi, I got unexpected behavior while testing
lucene. To shortly address> the problem: Using IndexWriter I add docs with fields named
ID with a> consecutive order (1,2,3,4, etc) then close that index. I get new> IndexReader,
and call IndexReader.deleteDocuments(Term). The term is> simply new Term("ID", "1"). and
then class close on IndexReader. Things> work out fine. But if i add docs using IndexWriter,
close writer, then> create new IndexReader to delete one of the docs already inserted,
but> without closing index. while the indexReader that perform deletion is> still not
closed, I add more docs, and then commit the IndexWriter, so> when i search I get all added
docs in the two phases (before using> deleteDocuments() on IndexReader and after because
i haven't closed> IndexReader, howerer, closed IndexWriter). I close IndexReader and then>
query the index, so i deletes all docs after opening it till closing it,> in addition to
the specified doc in the Term object (in this test case:> ID=1). I know that i can avoid
this by close IndexReader directly after> deleting docs, but what about runing it on mutiThread
app like web> application? There you are the code: > IndexSearcher indexSearcher = new
IndexSearcher(this.indexDirectory);> Hits hitsB4InsertAndClose = null;> hitsB4InsertAndClose
= getAllAsHits(indexSearcher);> int beforeInsertAndClose = hitsB4InsertAndClose.length();>
> indexWriter.addDocument(getNewElement());> indexWriter.addDocument(getNewElement());>
indexWriter.addDocument(getNewElement());> indexWriter.close();> IndexSearcher indexSearcherDel
= new IndexSearcher(this.indexDirectory);> indexSearcherDel.getIndexReader().deleteDocuments(new
Term("ID","1"));> > indexWriter.addDocument(getNewElement());> indexWriter.addDocument(getNewElement());>
indexWriter.addDocument(getNewElement());> indexWriter.addDocument(getNewElement());>
indexWriter.addDocument(getNewElement());> indexWriter.addDocument(getNewElement());>
indexWriter.addDocument(getNewElement());> indexWriter.addDocument(getNewElement());>
indexWriter.addDocument(getNewElement());> indexWriter.addDocument(getNewElement());>
indexWriter.addDocument(getNewElement());> > indexWriter.close();> Hits hitsAfterInsertAndClose
= getAllAsHits(indexSearcher);> int AfterInsertAndClose = hitsAfterInsertAndClose.length();//This
is 14> > indexWriter.addDocument(getNewElement());> indexWriter.close();> Hits
hitsAfterInsertAndAfterCloseb4Delete = getAllAsHits(indexSearcher);> int hitsAfterInsertButAndAfterCountb4Delete
=> hitsAfterInsertAndAfterCloseb4Delete.length();//This is 15> > > > indexSearcherDel.close();>
Hits hitsAfterInsertAndAfterClose = getAllAsHits(indexSearcher);int> hitsAfterInsertButAndAfterCount
=> hitsAfterInsertAndAfterClose.length();//This is 2 The two methods I> Use > private
Hits getAllAsHits(IndexSearcher indexSearcher){> try{> Analyzer analyzer = new StandardAnalyzer();>
String defaultSearchField = "all";> QueryParser parser = new QueryParser(defaultSearchField,
analyzer);> indexSearcher = new IndexSearcher(this.indexDirectory);> Hits hits = indexSearcher.search(parser.parse("+alias:mydoc"));>
indexSearcher.close();> return hits;> }catch(IOException ex){> throw new RuntimeException(ex);>
}catch(org.apache.lucene.queryParser.ParseException ex){> throw new RuntimeException(ex);>
}> > }> > private Document getNewElement(){> Map<String, String> map
= new HashMap();> map.put("ID", new Integer(insertCounter).toString());> map.put("name",
"name"+insertCounter);> insertCounter++;> Document document = new Document();> for
(Iterator iter = map.keySet().iterator(); iter.hasNext();) {> String key = (String) iter.next();>
document.add(new Field(key, map.get(key), Store.YES, Index.TOKENIZED));> }> document.add(new
Field("alias", "mydoc", Store.YES,> Index.UN_TOKENIZED));> return document;} any clue
why it works that way? I expect it to delete> only one doc? > _________________________________________________________________>
PC Magazine's 2007 editors' choice for best web mail-award-winning> Windows Live Hotmail.>
http://imagine-windowslive.com/hotmail/?locale=en-us&ocid=TXT_TAGHM_migr> ation_HMWL_mini_pcmag_0707>
> ---------------------------------------------------------------------> To unsubscribe,
e-mail: java-user-unsubscribe@lucene.apache.org> For additional commands, e-mail: java-user-help@lucene.apache.org>

_________________________________________________________________
See what you’re getting into…before you go there.
http://newlivehotmail.com
Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message