hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Harsh J <qwertyman...@gmail.com>
Subject Re: Question on Master
Date Wed, 16 Mar 2011 18:26:32 GMT
Yes, ${dfs.name.dir} is a NameNode used prop, while the other's a
DataNode used prop.

On Wed, Mar 16, 2011 at 11:41 PM, Mark <static.void.dev@gmail.com> wrote:
> Ok thanks for the clarification.
>
> Just to be sure though..
>
> - The master will have the ${dfs.name.dir} but not ${dfs.data.dir}
> - The nodes will have ${dfs.data.dir} but not ${dfs.name.dir}
>
> Is that correct?
>
> On 3/16/11 10:43 AM, Harsh J wrote:
>>
>> NameNode and JobTracker do not require a lot of storage space by
>> themselves. The NameNode needs some space to store its edits and
>> fsimage, and both require logging space.
>>
>> However, you may make use of multiple disks for NameNode, in order to
>> have a redundant backup copy of the NN image available in case one of
>> the disks crash. Other solutions to this second/third location include
>> storing to an HA-NFS mount, or an externally attached disk mount.
>>
>> NameNode data is stored in the ${dfs.name.dir} set of directories
>> [Defined in hdfs-site.xml]. Ensure its content is preserved perfectly
>> (including permissions) if you're planning to switch disks for your
>> NN.
>>
>> On Wed, Mar 16, 2011 at 10:04 PM, Mark<static.void.dev@gmail.com>  wrote:
>>>
>>> I know the master node is responsible for namenode and job tracker, but
>>> other than that is there any data stored on that machine? Basically what
>>> I
>>> am asking is should there be an generous amount of free space on that
>>> machine?
>>>
>>> So for example I have a large drive I want to swap out of my master and
>>> put
>>> into another machine which will be a used as a node. Before doing this,
>>> is
>>> there anything I should back up from my master? Where is the namenode
>>> data
>>> stored?
>>>
>>> Thanks
>>>
>>
>>
>



-- 
Harsh J
http://harshj.com

Mime
View raw message