hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Arun C Murthy (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HADOOP-1085) Remove 'port rolling' from Mini{DFS|MR}Cluster
Date Fri, 23 Mar 2007 08:52:32 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-1085?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12483466
] 

Arun C Murthy commented on HADOOP-1085:
---------------------------------------

@Konstantin
1. I tested this patch using HADOOP-1047 which seemed to solve TestReplication's failure,
it's out of sync with trunk now - anyway I've linked it to this issue to keep track.
3. The idea is to keep the dataNodeFirst (and equivalently taskTrackerFirst for MiniMRCluster)
so that people can manually run them if needed. However the automated test cases don't use
it; and hence that constructor isn't deprecated.
4. The sleep there is a busy wait till the namenode comes up. I agree with both of you that
we shouldn't use them to solve timing issues, which isn't the case here. We absolutely need
to wait for the NameNode to start so that we can then query it and start the datanodes. :)

@Jim
Jim, you wouldn't have to modify any config to get it to work, it is, as before, completely
automated (we just let the OS pick the ports). Maybe I misunderstand your concerns?

> Remove 'port rolling' from Mini{DFS|MR}Cluster
> ----------------------------------------------
>
>                 Key: HADOOP-1085
>                 URL: https://issues.apache.org/jira/browse/HADOOP-1085
>             Project: Hadoop
>          Issue Type: Bug
>          Components: test
>    Affects Versions: 0.12.0
>            Reporter: Arun C Murthy
>         Assigned To: Arun C Murthy
>             Fix For: 0.13.0
>
>         Attachments: HADOOP-1085_20070311_1.patch
>
>
> The rolling of ports in these 2 clusters lead to a lot of timing issues and failed test
cases; as witnessed in our patch process.
> The way around is to let the OS pick the port for the NameNode/JobTracker and let the
the DataNode/TaskTracker query them for the port to connect to and then use that port.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message