lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Vince Taluskie <vgtalus...@gmail.com>
Subject Re: Index Sizes
Date Tue, 17 May 2005 16:29:20 GMT
We're using a single dual-3Ghz Xeon box, Sun vx65 - indexes stored on Netapp 
nearstore R100. I think you can either try to investigate if there's a way 
your users will naturally group their searches and build indexes around that 
to minimize individual index size or prototype a distributed index search 
across enough processors to meet your service times. 

Hope that datapoint helps.... 

Vince

On 5/16/05, Richard Krenek <richard.krenek@gmail.com> wrote:
> 
> Unfortunately our indexes will be performance sensitive. Is Lucene
> still a good choice? What kind of hardware are you using?
> 
> Also what are the performance implications for having the additional
> 80 records in the index for just display purposes?
> 
> Thanks,
> Richard Krenek
> 
> 
> On 5/13/05, Vince Taluskie <vgtaluskie@gmail.com> wrote:
> > Yes, you'll be fine with 100 million, I've got a couple of 
> non-performance
> > sensitive indexes that are more than double that (280M) with about 20
> > seachable fields as well. We get results back in the 10-20 second range
> > which is fine for our end users.
> >
> > Vince
> >
> >
> > On 5/13/05, Richard Krenek <richard.krenek@gmail.com> wrote:
> > >
> > > Hypothetically I have 100 million records. Each record has 100+
> > > fields. Only 20 of those fields need to be searched on, the rest
> > > (including the 20) are just for display purposes.
> > > Would it be best to just add the 20 fields to the index and keep the
> > > rest in a relational database? What affect does all that fluff data
> > > have on the index size and search speeds? Does it matter that some of
> > > the fluff data is repeated a lot. (certain fields might just contain
> > > state a person lives, the color of their hair, number of fingers,
> > > etc).
> > > Our indexes are going to be very big, 100 million+ is not an
> > > exageration. Will Lucene handle this ok? I have created indexes in the
> > > 8-30 million range, but never this big in the number of documents and
> > > also the number of fields.
> > >
> > > Thanks for any info you can provide.
> > >
> > >
> > ---------------------------------------------------------------------
> > > To unsubscribe, e-mail:
> > java-user-unsubscribe@lucene.apache.org
> > > For additional commands, e-mail:
> > java-user-help@lucene.apache.org
> > >
> > >
> >
> >
> >
> > --
> >
> > @work
> > @home
> >
> > vince.taluskie (at) cexp.com <http://cexp.com> vince (at) taluskie.com<http://taluskie.com>
> > Corporate Express; Technical Architect Louisville, CO
> > Phone: 303 664 2660
> > http://www.taluskie.com
> >
> 



-- 

@work @home

vince.taluskie (at) cexp.com <http://cexp.com> vince (at)
taluskie.com<http://taluskie.com>
Corporate Express; Technical Architect Louisville, CO
Phone: 303 664 2660 http://www.taluskie.com

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message