hadoop-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jeff LI <uniquej...@gmail.com>
Subject Re: Host NameNode, DataNode, JobTracker or TaskTracker on the same machine
Date Thu, 14 Feb 2013 18:05:25 GMT
Thanks for your response.  I'm running SNN on another machine.

Could you explain a bit more on why I may run out of memory or disk?

I understand that NameNode holds file system metadata in memory.  I found
through this post that (
http://developer.yahoo.com/blogs/hadoop/posts/2010/05/scalability_of_the_hadoop_dist/
)
as a rule of thumb,
1 GB metadata ≈ 1 PB physical storage

Currently, my cluster has about 128TB of disk storage in total and 64GB
memory on each machine.  Does this suggests that I'm protected against
running out of memory from metadata?

Thanks

Cheers

Jeff


On Thu, Feb 14, 2013 at 12:41 PM, Tariq <dontariq@gmail.com> wrote:

> You may run out of memory,out of disk. If SNN is also running on the same
> machine then you are totally screwed in case of any breakdown
>
> shashwat shriparv <dwivedishashwat@gmail.com> wrote:
>
> >If you are doing it for production all the process should be running on
> >seperate machine as it will decrease the overload of the machine.
> >
> >
> >
> >∞
> >Shashwat Shriparv
> >
> >
> >
> >On Thu, Feb 14, 2013 at 10:40 PM, Jeff LI <uniquejeff@gmail.com> wrote:
> >
> >> Hello,
> >>
> >> Is there a good reason that we should not host NameNode, DataNode,
> >> JobTracker or TaskTracker services on the same machine?
> >>
> >> Not doing so is suggested here
> >http://wiki.apache.org/hadoop/NameNode,
> >> but I'd like to know the reasoning of this.
> >>
> >> Thanks
> >>
> >> Cheers
> >>
> >> Jeff
> >>
> >>
>
> --
> Sent from my Android phone with K-9 Mail. Please excuse my brevity.
>

Mime
View raw message