hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Aaron Kimball" <aa...@cloudera.com>
Subject Re: NameNode memory usage and 32 vs. 64 bit JVMs
Date Mon, 10 Nov 2008 09:30:32 GMT
Allen,

It sounds like you think the 64- and 32-bit environments are effectively
interchangable. May I ask why are you using both? The 64bit environment
gives you access to more memory; do you see faster performance for the TT's
in 32-bit mode? Do you get bit by library compatibility bugs that others
should watch out for in running a dual-mode Hadoop environment?

- Aaron

On Thu, Nov 6, 2008 at 10:49 PM, Allen Wittenauer <aw@yahoo-inc.com> wrote:

>
>
>
> On 11/6/08 10:17 PM, "C G" <parallelguy@yahoo.com> wrote:
>
> > I've got a grid which has been up and running for some time. It's been
> using a
> > 32 bit JVM.  I am hitting the wall on memory within NameNode and need to
> > specify max heap size > 4G.  Is it possible to switch seemlessly from
> 32bit
> > JVM to 64bit?  I've tried this on a small test grid and had no issues,
> but I
> > want to make sure it's OK to proceed.
>
>     It should be.  We run name node with 64-bit JVM and everything else
> with
> 32-bit.
>
> > Speaking of NameNode, what does it keep in memory?  Our memory usage
> ramped up
> > rather suddenly recently.
>
>     Some one/thing created a ton of files, likely small.  Check your file
> system contents, use the fs count command, etc, to look for anomalies.
>
> > Also, does SecondaryNameNode require the same
> > amount of memory as NameNode?
>
>     We're actually starting to see that it requires a tad bit more.
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message