hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Arun C Murthy (JIRA)" <j...@apache.org>
Subject [jira] Updated: (HADOOP-1085) Remove 'port rolling' from Mini{DFS|MR}Cluster
Date Sun, 11 Mar 2007 16:22:09 GMT

     [ https://issues.apache.org/jira/browse/HADOOP-1085?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Arun C Murthy updated HADOOP-1085:
----------------------------------

    Attachment: HADOOP-1085_20070311_1.patch

Here is a patch which removes 'port rolling' from the test-cases... a bit of a work in progress
since it seems to weirdly cause some streaming tests to fail.

Highlights:
1. Removes port-rolling by passing port '0' and letting the OS pick a ephemeral port.
2. Changed org.apache.hadoop.ipc.Server to bind in its constructor and not in the 'run' method
to be sure of the ephemeral port.
3. Since the 'port' changes from the one passed (which previously is stored in some data-members/configuration),
this resets them in all necessary places.
4. Removed support for 'dataNodeFirst/taskTrackerFirst' flag to the Mini{DFS|MR}Cluster since
they cannot be supported now (we can't know the NameNode/JobTracker ephemeral ports until
we start them).
5. Fixed all the test-cases and deprecated the unecessary constructors in Mini{DFS|MR}Cluster
which cannot be supported.


> Remove 'port rolling' from Mini{DFS|MR}Cluster
> ----------------------------------------------
>
>                 Key: HADOOP-1085
>                 URL: https://issues.apache.org/jira/browse/HADOOP-1085
>             Project: Hadoop
>          Issue Type: Bug
>          Components: test
>    Affects Versions: 0.12.0
>            Reporter: Arun C Murthy
>         Assigned To: Arun C Murthy
>             Fix For: 0.13.0
>
>         Attachments: HADOOP-1085_20070311_1.patch
>
>
> The rolling of ports in these 2 clusters lead to a lot of timing issues and failed test
cases; as witnessed in our patch process.
> The way around is to let the OS pick the port for the NameNode/JobTracker and let the
the DataNode/TaskTracker query them for the port to connect to and then use that port.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message