lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From ajay_garg <garg.jeena.isi.ka.naam....@gmail.com>
Subject Re: Concurrent Indexing + Searching
Date Mon, 04 Feb 2008 05:15:22 GMT

@Mark.

I am sorry, but I need a bit more of explanation. So you mean to say ::

"If auto-commit is false, then of course, docs will not be visible in the
index, until all the threads release themselves out of a particular
IndexWriter instance, and close() the IndexWriter instance.
If auto-commit is true, even then the above holds true. In particular, let's
say iI need an application 
with the following requirements ::

a) There are multiple indexer threads indexing on a SINGLE indexwriter
instance with auto-commit true
b) Each thread 'flushes' according to a pre-defined criteria at some point
of time.
c) The index should be updated immediately, that is, if any user re-opens
the IndexSearcher, then the 
    documents added till-that-snapshot-of-index must be visible. Note that
the IndexWriter instance hasn't 
    been closed as yet, the indexer threads will be indexing till eternity,
so that IndexWriter instance will 
    never be closed.

So, you presume that building an application with the above requirements is
impossible, even with auto-commit set to true. "

( If I sound ambiguous at any point, kindly forgive me for my lack of
language skills. I will try to explain better, if need arises ).

Looking forward to a reply
Ajay Garg

markrmiller wrote:
> 
> You are correct that autocommit=false means that docs will be in the 
> index before the last thread releases its concurrent hold on a Writer, 
> *but because IndexAccessor controls* *when the IndexSearchers are 
> reopened*, those docs will still not be visible until the last thread 
> holding a Writer releases it...that is when the reopening of Searchers 
> occurs as well as when the Writer is closed.
> 
> - Mark
> 
> ajay_garg wrote:
>> Hi. Sorry if I seem a stranger in this thread, but there is something
>> that I
>> can't resist clearing myself on.
>>
>> Mark, you say that the additional documents added to a index, won't show
>> up
>> until the # of threads accessing the index hits 0; and subsequently the
>> indexwriter instance is closed.
>>
>> But I suppose that the autocommit=true, asserts that all flushed (Added)
>> documents are immediately committed ( and hence visible ) in the index,
>> and
>> no explicit cclosing ( releasiing ) of the Indexwriter instance is
>> required.
>> ( Of course, re-opening an IndexSearcher instance is required ).
>>
>> Am I being dumb ?
>>
>> Looking eagerly for you to shed some light on my doubt.
>>
>> Thanks
>> Ajay Garg
>>
>>
>> codetester wrote:
>>   
>>> Hi All,
>>>
>>> A newbie out here.... I am using lucene 2.3.0. I need to use lucene to
>>> perform live searching and indexing. To achieve that, I tried the
>>> following
>>>
>>> FSDirectory directory = FSDirectory.getDirectory(location);
>>> IndexReader reader = IndexReader.open(directory );
>>> IndexWriter writer = new IndexWriter(directory , new SimpleAnalyzer(),
>>> true); // <- I want to recreate the index every time
>>> IndexSearcher searcher = new IndexSearcher( reader );
>>>
>>> For Searching, I have the following code
>>> QueryParser queryParser = new QueryParser("xyz", new
>>> StandardAnalyzer());
>>> Hits hits = searcher .search(queryParser.parse(displayName + "*"));
>>>
>>> And for adding records, I have the following code
>>>  // Create doc object
>>>  writer.addDocument(doc);
>>>
>>>  IndexReader newIndexReader = reader.reopen() ;
>>>  if ( newIndexReader != reader ) {
>>>        reader.close() ;
>>>  }
>>>  reader = newIndexReader ;
>>>  searcher.close() ;
>>>  searcher = new IndexSearcher(reader );
>>>         
>>> So the issues that I face are 
>>>
>>> 1) The addition of new record is not reflected in the search ( even
>>> though
>>> I have reinited IndexSearcher )
>>>
>>> 2) Obviously, the add record code is not thread safe. I am trying to
>>> close
>>> and update the reference to IndexSearcher object. I could add a sync
>>> block, but the bigger question would be that what is the ideal way to
>>> achieve this case where I need to add and search record real-time ? 
>>>
>>> Thanks !
>>>
>>>
>>>
>>>
>>>
>>>     
>>
>>   
> 
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
> 
> 
> 

-- 
View this message in context: http://www.nabble.com/Concurrent-Indexing-%2B-Searching-tp15234463p15262305.html
Sent from the Lucene - Java Users mailing list archive at Nabble.com.


---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org


Mime
View raw message