lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Alex Murzaku" <li...@lissus.com>
Subject RE: Indexing Growth
Date Wed, 02 Apr 2003 14:22:07 GMT
Hi Rob,

After you build the initial index (10.4M) you can see that the directory
contains some files - those are the index files. I am sure that the size
and number of those initial files doesn't change while you query Lucene.
When you say the index grows, I assume you are referring to other files
that are added to the index directory therefore increasing the size of
the directory and not the initial index files. One experiment you could
do is to inspect the index directory after each query. This should give
you an idea of what is going on. Also, it might help sending some code
snippets to the list to get more informed help...

Alex

-----Original Message-----
From: Rob Outar [mailto:routar@ideorlando.org] 
Sent: Wednesday, April 02, 2003 8:51 AM
To: Lucene Users List
Subject: RE: Indexing Growth
Importance: High


Hi all,

	This is too odd and I do not even know where to start.  We built
a Windows Explorer type tool that indexes all files in a "sabdboxed"
file system. Each Lucene document contains stuff like path, parent
directory, last modified date, file_lock etc..  When we display the
files in a given directory through the tool we query the index about 5
times for each file in the repository, this is done so we can display
all attributes in the index about that file.  So for example if there
are 5 files in the directory, each file has 6 attributes that means
about 30 term queries are executed.  The initial index when build it
about 10.4megs, after accessing about 3 or 4 directories the index size
increased to over 100megs, and we did not add anything!!  All we are
doing is querying!!  Yesterday after querying became ungodly slow, we
looked at the index size it had grown from 10megs to 1.5GB (granted we
tested the tool all morning).  But I have no idea why the index is
growing like this.  ANY help would be greatly appreciated.


Thanks,

Rob


-----Original Message-----
From: Rob Outar [mailto:routar@ideorlando.org]
Sent: Tuesday, April 01, 2003 3:32 PM
To: Lucene Users List; lists@lissus.com
Subject: RE: Indexing Growth


I reuse the same searcher, analyzer and Query object I don't think that
should cause the problem.

Thanks,

Rob


-----Original Message-----
From: Alex Murzaku [mailto:lists@lissus.com]
Sent: Tuesday, April 01, 2003 3:22 PM
To: 'Lucene Users List'
Subject: RE: Indexing Growth


I don't know if I remember this correctly: I think for every query
(term) is created a file but the file should disappear after the query
is completed.

-----Original Message-----
From: Rob Outar [mailto:routar@ideorlando.org]
Sent: Tuesday, April 01, 2003 3:13 PM
To: Lucene Users List
Subject: RE: Indexing Growth


Dang I must be doing something crazy cause all my client app does is
search and the index size increases.  I do not add anything.

Thanks,

Rob


-----Original Message-----
From: Otis Gospodnetic [mailto:otis_gospodnetic@yahoo.com]
Sent: Tuesday, April 01, 2003 3:07 PM
To: Lucene Users List
Subject: Re: Indexing Growth


Only when you add new documents to it.

Otis

--- Rob Outar <routar@ideorlando.org> wrote:
> Hi all,
>
> 	Will the index grow based on queries alone?  I build my index,
then
> run several queries against it and afterwards I check the size of the 
> index and in some cases it has grown quite a bit although I did not 
> add anything???
>
> Anyhow please let me know the cases when the index will grow.
>
> Thanks,
>
> Rob
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: lucene-user-unsubscribe@jakarta.apache.org
> For additional commands, e-mail: lucene-user-help@jakarta.apache.org
>


__________________________________________________
Do you Yahoo!?
Yahoo! Tax Center - File online, calculators, forms, and more
http://platinum.yahoo.com

---------------------------------------------------------------------
To unsubscribe, e-mail: lucene-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: lucene-user-help@jakarta.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: lucene-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: lucene-user-help@jakarta.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: lucene-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: lucene-user-help@jakarta.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: lucene-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: lucene-user-help@jakarta.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: lucene-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: lucene-user-help@jakarta.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: lucene-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: lucene-user-help@jakarta.apache.org


Mime
View raw message