hadoop-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Harsh J <ha...@cloudera.com>
Subject Re: rack topology data update
Date Thu, 13 Sep 2012 08:03:45 GMT
Hi Jameson,

As I'd mentioned, due to the current behavior, if the NN has cached a
bad topology mapping value already, it will not forget it despite a
-refreshNodes command. Otherwise, there's no problem. That is if
you've done the following, NN may require a restart:

1. Start new DN (At this point, DN gets mapped to default rack as
there's no entry, and this is cached)
2. Update topology file, do refreshNodes

This should be fixed in one of the 2.x releases, where we also refresh
the cached values.

On Thu, Sep 13, 2012 at 12:49 PM, Jameson Li <hovlj.ei@gmail.com> wrote:
> Hi  Harsh J,  Viji R,  Saurabh bhutyani,
>
> Thanks for all of yours replying.
>
> But really the namenode not refresh the rack info.
>
> Is my hadoop version issue? My hadoop version is base on hadoop-0.20-append,
> and have 4 patches on it that I think they are really no matter with the
> rack awareness.
>
> If anytime I add new nodes, and I should restart namenode to refresh the
> rack info, I will crazy...
>
> 专注于Mysql,MSSQL,Oracle,Hadoop
>
>
>
> 2012/9/13 Saurabh bhutyani <s4saurabh@gmail.com>
>>
>> I believe running the following command on namenode should refresh it.
>>
>> 'hadoop dfsadmin -refreshNodes'
>>
>> Thanks & Regards,
>> Saurabh Bhutyani
>>
>> Call  : 9820083104
>> Gtalk: s4saurabh@gmail.com
>>
>>
>>
>>
>> On Thu, Sep 13, 2012 at 11:25 AM, Viji R <viji@cloudera.com> wrote:
>>>
>>> Hi Jameson,
>>>
>>> If the NameNode has cached the wrong value earlier, it will not
>>> refresh that until you restart it.
>>>
>>> On Thu, Sep 13, 2012 at 11:21 AM, Jameson Li <hovlj.ei@gmail.com> wrote:
>>> > Hi harsh,
>>> >
>>> > I have followed your suggestion operation.
>>> >
>>> > 1, stop the new datanode.(I have modified the topology file in the
>>> > namenode
>>> > before.)
>>> > 2, run 'hadoop dfsadmin -refreshNodes' on the namenode
>>> > 3, start the new datanode.
>>> >
>>> > But it really not update the new topology mapping.
>>> > It just show the start info in the namenode that:
>>> > "
>>> > 2012-09-13 13:44:14,706 INFO org.apache.hadoop.net.NetworkTopology:
>>> > Removing
>>> > a node: /default-rack/10.0.10.100:50010
>>> > 2012-09-13 13:44:14,706 INFO org.apache.hadoop.net.NetworkTopology:
>>> > Adding a
>>> > new node: /default-rack/10.0.10.100:50010
>>> > "
>>> >
>>> >
>>> > 专注于Mysql,MSSQL,Oracle,Hadoop
>>> >
>>> >
>>> >
>>> > 2012/9/13 Harsh J <harsh@cloudera.com>
>>> >>
>>> >> the DN only after (2) so it picks up the right mapping an
>>> >
>>> >
>>
>>
>



-- 
Harsh J

Mime
View raw message