lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Carlos Pita" <carlosjosep...@gmail.com>
Subject Re: maxDoc and arrays
Date Fri, 25 May 2007 06:59:17 GMT
I see. Anyway I would update the array when adding a document, so my reader
would be closed then, and just a writer would be accessing the index.
Supposing that no merging is triggered (for this I'm choosing a big
mergeFactor and forcing optimization when a number of documents has been
added) the numeration will be kept.

OTOH, I do some tests with FieldCache too. As I have to keep a number of ids
(an array of shorts) for each document, I tried marshaling them into a
String so they can be retrieved from the cache later. The performance is far
better than that for directly retrieving documents but still notably behind
the array approach. Maybe this is due to the unmarshaling step, I'm still
unsure.

I will try an hybrid approach where the FieldCache stores just an int which
is itself the index to the array with the effective ids. This way I won't
have to deal with index renumbering and at the same time will keep most of
the data in memory in its definitive format.

Thank you for your answer.
Cheers,
Carlos

On 5/25/07, Antony Bowesman <adb@teamware.com> wrote:
>
> Carlos Pita wrote:
> > Hi all,
> >
> > Is there any guaranty that the maxDoc returned by a reader will be about
> > the
> > total number of indexed documents?
>
> It struck me in this thread was that there may be a misunderstanding of
> the
> relationship between numDocs/maxDoc and an IndexReader.
>
> When an IndexReader is opened its maxDoc and numDocs will never change
> regardless of the additions or deletions to the index.  At least I've not
> been
> able to make them change in my test cases.
>
> So, when adding a new document after a reader has been opened, this new
> document
> is not yet visible via the original reader, so if you are caching that
> array,
> you would not update that array as it relates to the reader on the index
> at the
> time the reader was opened.
>
> When you open a new reader, the numDocs and maxDoc will reflect that
> addition.
> Same applies to deletions.  After opening the reader, you would need to
> regenerate you array cache.
>
> As Hoss has said, this is pretty much what FieldCache does and it holds
> the
> caches keyed by the IndexReader.
>
> Antony
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message