lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From John Moylan <jo...@rte.ie>
Subject Re: Large index files
Date Fri, 23 Jul 2004 13:37:48 GMT
Depends... I index everything to ram, optimize and then dump to the
disk. I end up with three files:

segments
deletable
_1.cfs

John

On Fri, 2004-07-23 at 14:38, Joel Shellman wrote:
> I'm a little confused by this. I thought Lucene keeps creating new files 
> as the index gets bigger and any single file doesn't ever get all that 
> big. Is that not the case?
> 
> Thanks,
> 
> Joel Shellman
> 
> 
> John Moylan wrote:
> > As long as your kernel has "Large File Support", then you should be
> > fine. Most modern distro's support >2GB files now out of the box.
> > 
> > John 
> > 
> > On Fri, 2004-07-23 at 13:44, Karthik N S wrote:
> > 
> >>Hi
> >>
> >>  I think  (a) would be a better choice  [I have  done it on Linux  upt to
> >>7GB , it's pretty faster then doing the same on win2000 PF]
> >>
> >>
> >>with regards
> >>Karthik
> >>
> >>-----Original Message-----
> >>From: Rupinder Singh Mazara [mailto:rsmazara@ebi.ac.uk]
> >>Sent: Friday, July 23, 2004 5:55 PM
> >>To: Lucene Users List
> >>Subject: Large index files
> >>
> >>
> >>Hi all
> >>
> >>  I am using lucene to index a large dataset, it so happens 10% of this data
> >>yields indexes of
> >>  400MB, in all likelihood it is possible the index may go upto 7GB.
> >>
> >>  My deployment will be on a linux/tomcat  system, what will be a better
> >>solution
> >>  a) create one large index and hope linux does not mind
> >>  b) generate 7-10 indexes based on some criteria and glue them together
> >>using MultiReader, in this case I may cross the MAX file handles limit of
> >>Tomcat ?
> >>
> >> regards
> >>
> >>
> >>
> >>
> >>
> >>
> >>
> >>---------------------------------------------------------------------
> >>To unsubscribe, e-mail: lucene-user-unsubscribe@jakarta.apache.org
> >>For additional commands, e-mail: lucene-user-help@jakarta.apache.org
> >>
> >>
> >>---------------------------------------------------------------------
> >>To unsubscribe, e-mail: lucene-user-unsubscribe@jakarta.apache.org
> >>For additional commands, e-mail: lucene-user-help@jakarta.apache.org
> 
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: lucene-user-unsubscribe@jakarta.apache.org
> For additional commands, e-mail: lucene-user-help@jakarta.apache.org
-- 
John Moylan
RTÉ ePublishing,
Montrose House,
Donnybrook,
Dublin 4
T: +353 1 2083564
E: john.moylan@rte.ie


******************************************************************************
The information in this e-mail is confidential and may be legally privileged.
It is intended solely for the addressee. Access to this e-mail by anyone else
is unauthorised. If you are not the intended recipient, any disclosure,
copying, distribution, or any action taken or omitted to be taken in reliance
on it, is prohibited and may be unlawful.
Please note that emails to, from and within RTÉ may be subject to the Freedom
of Information Act 1997 and may be liable to disclosure.
******************************************************************************

---------------------------------------------------------------------
To unsubscribe, e-mail: lucene-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: lucene-user-help@jakarta.apache.org


Mime
View raw message