lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Rob Outar" <rou...@ideorlando.org>
Subject RE: Indexing Growth
Date Wed, 02 Apr 2003 14:50:19 GMT
After building the index for the first time:

_l1d.f1  _l1d.f3  _l1d.f5  _l1d.f7  _l1d.f9   _l1d.fdx  _l1d.frq  _l1d.tii
deletable
_l1d.f2  _l1d.f4  _l1d.f6  _l1d.f8  _l1d.fdt  _l1d.fnm  _l1d.prx  _l1d.tis
segments

After running first query to get all attributes from all files in the given
directory, there were 17 files, each file has 5 attributes so 85 queries
were ran:

_l1j.f1   _l1p.f9   _l21.f3   _l27.fdx  _l2j.f5   _l2p.prx  _l31.f7
_l3j.f1   _l3p.f9   _l41.f3   _l44.fdx
_l1j.f2   _l1p.fdt  _l21.f4   _l27.frq  _l2j.f6   _l2p.tis  _l31.f8
_l3j.f2   _l3p.fdt  _l41.f4   _l44.frq
_l1j.f3   _l1p.fdx  _l21.f5   _l27.prx  _l2j.f7   _l2v.f1   _l31.f9
_l3j.f3   _l3p.fdx  _l41.f5   _l44.prx
_l1j.f4   _l1p.frq  _l21.f6   _l27.tis  _l2j.f8   _l2v.f2   _l31.fdt
_l3j.f4   _l3p.frq  _l41.f6   _l44.tis
_l1j.f5   _l1p.prx  _l21.f7   _l2d.f1   _l2j.f9   _l2v.f3   _l31.fdx
_l3j.f5   _l3p.prx  _l41.f7   _l47.f1
_l1j.f6   _l1p.tis  _l21.f8   _l2d.f2   _l2j.fdt  _l2v.f4   _l31.frq
_l3j.f6   _l3p.tis  _l41.f8   _l47.f2
_l1j.f7   _l1v.f1   _l21.f9   _l2d.f3   _l2j.fdx  _l2v.f5   _l31.prx
_l3j.f7   _l3v.f1   _l41.f9   _l47.f3
_l1j.f8   _l1v.f2   _l21.fdt  _l2d.f4   _l2j.frq  _l2v.f6   _l31.tis
_l3j.f8   _l3v.f2   _l41.fdt  _l47.f4
_l1j.f9   _l1v.f3   _l21.fdx  _l2d.f5   _l2j.prx  _l2v.f7   _l37.f1
_l3j.f9   _l3v.f3   _l41.fdx  _l47.f5
_l1j.fdt  _l1v.f4   _l21.frq  _l2d.f6   _l2j.tis  _l2v.f8   _l37.f2
_l3j.fdt  _l3v.f4   _l41.frq  _l47.f6
_l1j.fdx  _l1v.f5   _l21.prx  _l2d.f7   _l2p.f1   _l2v.f9   _l37.f3
_l3j.fdx  _l3v.f5   _l41.prx  _l47.f7
_l1j.frq  _l1v.f6   _l21.tis  _l2d.f8   _l2p.f2   _l2v.fdt  _l37.f4
_l3j.frq  _l3v.f6   _l41.tis  _l47.f8
_l1j.prx  _l1v.f7   _l27.f1   _l2d.f9   _l2p.f3   _l2v.fdx  _l37.f5
_l3j.prx  _l3v.f7   _l44.f1   _l47.f9
_l1j.tis  _l1v.f8   _l27.f2   _l2d.fdt  _l2p.f4   _l2v.frq  _l37.f6
_l3j.tis  _l3v.f8   _l44.f2   _l47.fdt
_l1p.f1   _l1v.f9   _l27.f3   _l2d.fdx  _l2p.f5   _l2v.prx  _l37.f7
_l3p.f1   _l3v.f9   _l44.f3   _l47.fdx
_l1p.f2   _l1v.fdt  _l27.f4   _l2d.frq  _l2p.f6   _l2v.tis  _l37.f8
_l3p.f2   _l3v.fdt  _l44.f4   _l47.fnm
_l1p.f3   _l1v.fdx  _l27.f5   _l2d.prx  _l2p.f7   _l31.f1   _l37.f9
_l3p.f3   _l3v.fdx  _l44.f5   _l47.frq
_l1p.f4   _l1v.frq  _l27.f6   _l2d.tis  _l2p.f8   _l31.f2   _l37.fdt
_l3p.f4   _l3v.frq  _l44.f6   _l47.prx
_l1p.f5   _l1v.prx  _l27.f7   _l2j.f1   _l2p.f9   _l31.f3   _l37.fdx
_l3p.f5   _l3v.prx  _l44.f7   _l47.tii
_l1p.f6   _l1v.tis  _l27.f8   _l2j.f2   _l2p.fdt  _l31.f4   _l37.frq
_l3p.f6   _l3v.tis  _l44.f8   _l47.tis
_l1p.f7   _l21.f1   _l27.f9   _l2j.f3   _l2p.fdx  _l31.f5   _l37.prx
_l3p.f7   _l41.f1   _l44.f9   deletable
_l1p.f8   _l21.f2   _l27.fdt  _l2j.f4   _l2p.frq  _l31.f6   _l37.tis
_l3p.f8   _l41.f2   _l44.fdt  segments

I have no reason to add anything to the index all I want to do is getch the
attributes for the list of files in that directory.

Thanks,

Rob


-----Original Message-----
From: Ian Lea [mailto:ian@digimem.net]
Sent: Wednesday, April 02, 2003 9:24 AM
To: Rob Outar
Cc: Lucene Users List
Subject: RE: Indexing Growth


What does the index directory look like before and after running
queries?  Are files growing or being added?  Which files? How many
documents are there in the index before and after? Are you absolutely
100% positive there is no way that your application is adding entries
to the index?  That still has to be the most likely explanation, I think.



--
Ian.
ian@digimem.net


> routar@ideorlando.org (Rob Outar) wrote
>
> Hi all,
>
> 	This is too odd and I do not even know where to start.  We built a
Windows
> Explorer type tool that indexes all files in a "sabdboxed" file system.
> Each Lucene document contains stuff like path, parent directory, last
> modified date, file_lock etc..  When we display the files in a given
> directory through the tool we query the index about 5 times for each file
in
> the repository, this is done so we can display all attributes in the index
> about that file.  So for example if there are 5 files in the directory,
each
> file has 6 attributes that means about 30 term queries are executed.  The
> initial index when build it about 10.4megs, after accessing about 3 or 4
> directories the index size increased to over 100megs, and we did not add
> anything!!  All we are doing is querying!!  Yesterday after querying
became
> ungodly slow, we looked at the index size it had grown from 10megs to
1.5GB
> (granted we tested the tool all morning).  But I have no idea why the
index
> is growing like this.  ANY help would be greatly appreciated.
>
>
> Thanks,
>
> Rob
>
>
> -----Original Message-----
> From: Rob Outar [mailto:routar@ideorlando.org]
> Sent: Tuesday, April 01, 2003 3:32 PM
> To: Lucene Users List; lists@lissus.com
> Subject: RE: Indexing Growth
>
>
> I reuse the same searcher, analyzer and Query object I don't think that
> should cause the problem.
>
> Thanks,
>
> Rob
>
>
> -----Original Message-----
> From: Alex Murzaku [mailto:lists@lissus.com]
> Sent: Tuesday, April 01, 2003 3:22 PM
> To: 'Lucene Users List'
> Subject: RE: Indexing Growth
>
>
> I don't know if I remember this correctly: I think for every query
> (term) is created a file but the file should disappear after the query
> is completed.
>
> -----Original Message-----
> From: Rob Outar [mailto:routar@ideorlando.org]
> Sent: Tuesday, April 01, 2003 3:13 PM
> To: Lucene Users List
> Subject: RE: Indexing Growth
>
>
> Dang I must be doing something crazy cause all my client app does is
> search and the index size increases.  I do not add anything.
>
> Thanks,
>
> Rob
>
>
> -----Original Message-----
> From: Otis Gospodnetic [mailto:otis_gospodnetic@yahoo.com]
> Sent: Tuesday, April 01, 2003 3:07 PM
> To: Lucene Users List
> Subject: Re: Indexing Growth
>
>
> Only when you add new documents to it.
>
> Otis
>
> --- Rob Outar <routar@ideorlando.org> wrote:
> > Hi all,
> >
> > 	Will the index grow based on queries alone?  I build my index,
> then
> > run several queries against it and afterwards I check the size of the
> > index and
> > in some cases it has grown quite a bit although I did not add
> > anything???
> >
> > Anyhow please let me know the cases when the index will grow.
> >
> > Thanks,
> >
> > Rob
> >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: lucene-user-unsubscribe@jakarta.apache.org
> > For additional commands, e-mail: lucene-user-help@jakarta.apache.org
> >
>
>
> __________________________________________________
> Do you Yahoo!?
> Yahoo! Tax Center - File online, calculators, forms, and more
> http://platinum.yahoo.com
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: lucene-user-unsubscribe@jakarta.apache.org
> For additional commands, e-mail: lucene-user-help@jakarta.apache.org
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: lucene-user-unsubscribe@jakarta.apache.org
> For additional commands, e-mail: lucene-user-help@jakarta.apache.org
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: lucene-user-unsubscribe@jakarta.apache.org
> For additional commands, e-mail: lucene-user-help@jakarta.apache.org
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: lucene-user-unsubscribe@jakarta.apache.org
> For additional commands, e-mail: lucene-user-help@jakarta.apache.org
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: lucene-user-unsubscribe@jakarta.apache.org
> For additional commands, e-mail: lucene-user-help@jakarta.apache.org
>
>
----------------------------------------------------------------------
Searchable personal storage and archiving from http://www.digimem.net/



---------------------------------------------------------------------
To unsubscribe, e-mail: lucene-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: lucene-user-help@jakarta.apache.org


Mime
View raw message