lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From James Dunn <james_h_d...@yahoo.com>
Subject Re: Compound file format file size question
Date Fri, 18 Jun 2004 18:11:46 GMT
Otis,

Thanks for the response.

Yeah, I was copying the file to a brand new hard-drive
and it was formatted to FAT32 by default, which is
probably why it couldn't handle the 13GB file.  

I'm converting the drive to NTFS now, which should get
me through temporarily.  In the future though, I may
break the index up into smaller sub-indexes so that I
can distribute them across seperate physical disks for
better disk IO.

Thanks for your help!

Jim
--- Otis Gospodnetic <otis_gospodnetic@yahoo.com>
wrote:
> Hello,
> 
> --- James Dunn <james_h_dunn@yahoo.com> wrote:
> > Hello all,
> > 
> > I have an index that's about 13GB on disk.  I'm
> using
> > 1.4 rc3 which uses the compound file format by
> > default.
> > 
> > Once I run optimize on my index, it creates one
> 13GB
> > ..cfs file.  This isn't a problem on Linux (yet),
> but
> > I'm having some trouble copying the file over to
> my
> > Windows XP box.
> 
> What is the exact problem? The sheer size of it or
> something else? 
> Just curious...
> 
> > Is there some way using the compound file format
> to
> > set the maximum file size and have Lucene break
> the
> > index into multiple files once it hits that limit?
> 
> Can't be done with Lucene, but I seem to recall some
> discussion about
> it.  Nothing concrete, though.
> 
> > Or do I need to go back to using the non-compound
> file
> > format?
> 
> The total size should be (about) the same, but you
> could certainly do
> that, if having more smaller files is better for
> you.
> 
> Otis
> 
> > Another solution, I suppose, would be to break up
> my
> > index into seperate smaller indexes.  This would
> be my
> > second choice, however.
> > 
> > Thanks a lot,
> > 
> > Jim
> 
> 
>
---------------------------------------------------------------------
> To unsubscribe, e-mail:
> lucene-user-unsubscribe@jakarta.apache.org
> For additional commands, e-mail:
> lucene-user-help@jakarta.apache.org
> 
> 


---------------------------------------------------------------------
To unsubscribe, e-mail: lucene-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: lucene-user-help@jakarta.apache.org


Mime
View raw message