lucene-solr-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Antonio Eggberg <antonio_eggb...@yahoo.se>
Subject Re: how large can the index be?
Date Mon, 29 Dec 2008 21:19:23 GMT
Thanks you very much for your answer.

I was afraid of that the each document has about 20 fields.. As you pointed out it will slow
down. Anyway I am thinking is it not possible to do the following:

Load Balancer 
     |
Solr A, Solr B, ...
     |
  one index

So I send 50% query to Solr A, 50% to Solr B and so forth.. is this not good? Also to add
The index will be like a mounted drive to the solr boxes... On the above do I really need
to worry about Solr Master, Solr Slave? It probably solve my load but I think query speed
will be slow...

Just curious anyone using distributed search in production?

Cheers



--- Den mån 2008-12-29 skrev Otis Gospodnetic <otis_gospodnetic@yahoo.com>:

> Från: Otis Gospodnetic <otis_gospodnetic@yahoo.com>
> Ämne: Re: how large can the index be?
> Till: solr-user@lucene.apache.org
> Datum: måndag 29 december 2008 21.53
> Hi Antonio,
> 
> Besides thinking in terms of documents, you also need to
> think in terms of index size on the file system vs. the
> amount of RAM your search application/server can use.  50M
> documents may be doable on a single server if those
> documents are not too large and you have sufficient RAM.  It
> gets even better if your index doesn't change very often
> and if you can get decent hit ratios on the various Solr
> caches.
> 
> If you are indexing largish documents, or even something as
> small as an average web page, 50M docs may be too much on a
> "commodity box" (say dual core 8 GB RAM box)
> 
> Otis
> --
> Sematext -- http://sematext.com/ -- Lucene - Solr - Nutch
> 
> 
> 
> ----- Original Message ----
> > From: Antonio Eggberg <antonio_eggberg@yahoo.se>
> > To: solr-user@lucene.apache.org
> > Sent: Monday, December 29, 2008 3:41:48 PM
> > Subject: how large can the index be?
> > 
> > Hi,
> > 
> > We are running successfully a solr index of 3 million
> docs. I have just been 
> > informed that our index size will increase to 50
> million. I been going through 
> > the doc 
> > 
> > http://wiki.apache.org/solr/DistributedSearch
> > 
> > Seems like we will loose out on the date facet and
> some more other stuff that we 
> > use. which is important to us. So far we been using 1
> index and 1 machine. 
> > 
> > Can I still stick with my 1 index but have many query
> servers? We don't update 
> > our index so often this are rather static data. Over
> the past year we have 
> > updated the index data a total of 3 times and about
> 300 records :)
> > 
> > Can someone provide some idea how/what should I do to
> deal with new datasets?.
> > 
> > Thanks for your help.
> > 
> > 
> >      
> __________________________________________________________
> > Går det långsamt? Skaffa dig en snabbare
> bredbandsuppkoppling. 
> > Sök och jämför priser hos Kelkoo.
> >
> http://www.kelkoo.se/c-100015813-bredband.html?partnerId=96914325


      __________________________________________________________
Låna pengar utan säkerhet. Jämför vilkor online hos Kelkoo.
http://www.kelkoo.se/c-100390123-lan-utan-sakerhet.html?partnerId=96915014

Mime
View raw message