lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Erick Erickson <erickerick...@gmail.com>
Subject Re: Scale up design
Date Mon, 13 Dec 2010 13:30:06 GMT
Here's a great intro to the garbage collection options:
http://www.lucidimagination.com/blog/2009/09/19/java-garbage-collection-boot-camp-draft/

@Ganesh:
The issue with 64 bit isn't really performance, it's that you can't allocate
much of
your memory to the JVM. So by definition your performance will tank much
earlier
with a 32-bit JVM no matter what physical hardware you're running on because
you're constrained by how much of the physical memory you *can* use.

Best
Erick

On Mon, Dec 13, 2010 at 6:33 AM, William Newport <billy@billynewport.com>wrote:

> I've used 30-35gb heaps and it is painful.
>
> Sent from my iPhone
>
> On Dec 13, 2010, at 2:16 AM, "Danil Ε’ORIN" <torindan@gmail.com> wrote:
>
> > GC times on large heaps are pretty painfull right now (haven't tried
> > G1 collector, knowledgeable people : please advise)
> >
> > Also it's very dependent on your index and query pattern, so you could
> > improve it by using some -XX magic.
> >
> > My recommendation is to scale horizontally (spit index into shards),
> > this way you'll be able to scale up much easier than moving to even
> > beefier server.
> > Initially if your server is big enough, you may host all your shards
> > on it, just in separate jvms.
> >
> > If you are thinking on BIG indexes, you probably don't want to loose
> > them, so you also must think of replication, standbys and so on.
> > And from my experience overall cost (for same availability) is cheaper
> > when you use many smaller servers than few large ones.
> >
> > On Mon, Dec 13, 2010 at 09:01, Ganesh <emailgane@yahoo.co.in> wrote:
> >> Have you tried using Lucene in 64 Bit with more than 8 GB RAM.
> >>
> >> Regards
> >> Ganesh
> >>
> >> ----- Original Message -----
> >> From: "Pradeep Singh" <pksinghus@gmail.com>
> >> To: <java-user@lucene.apache.org>
> >> Sent: Monday, December 13, 2010 12:16 PM
> >> Subject: Re: Scale up design
> >>
> >>
> >>> 8GB is used on laptops. For servers you need more.
> >>>
> >>> On Sun, Dec 12, 2010 at 10:25 PM, Ganesh <emailgane@yahoo.co.in>
> wrote:
> >>>
> >>>> Hello all,
> >>>>
> >>>> I know little bit about scale out design, Sharding the database across
> >>>> systems. Is any one in this group tried Scale up architecture? I think
> to
> >>>> scale up, we need to use 64 bit. How about the Lucene performance in
> 64 bit?
> >>>> Whether we could use 8 GB RAM completely?
> >>>>
> >>>> Could any share their thoughts on this.
> >>>>
> >>>> Regards
> >>>> Ganesh
> >>>> Send free SMS to your Friends on Mobile from your Yahoo! Messenger.
> >>>> Download Now! http://messenger.yahoo.com/download.php
> >>>>
> >>>> ---------------------------------------------------------------------
> >>>> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> >>>> For additional commands, e-mail: java-user-help@lucene.apache.org
> >>>>
> >>>>
> >>>
> >> Send free SMS to your Friends on Mobile from your Yahoo! Messenger.
> Download Now! http://messenger.yahoo.com/download.php
> >>
> >> ---------------------------------------------------------------------
> >> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> >> For additional commands, e-mail: java-user-help@lucene.apache.org
> >>
> >>
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> > For additional commands, e-mail: java-user-help@lucene.apache.org
> >
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message