hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Nikhil S. Ketkar (Updated) (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (MAPREDUCE-3315) Master-Worker Application on YARN
Date Wed, 04 Apr 2012 18:33:24 GMT

     [ https://issues.apache.org/jira/browse/MAPREDUCE-3315?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Nikhil S. Ketkar updated MAPREDUCE-3315:
----------------------------------------

    Attachment: MAPREDUCE-3315-2.patch

Patch contains previous implementation with the hard coding of the port for the Hadoop IPC
(in the Master and Workers) removed. 

Now we iterate over the Linux Ephemeral Port range from 32768 to 61000. Catch the BindException
implying that the port is being used. Then move on to the next port. If no port is found from
32768 to 61000 throw BindException. 

We keep track of the port and send it to the Workers (Containers) as an environment variable.
                
> Master-Worker Application on YARN
> ---------------------------------
>
>                 Key: MAPREDUCE-3315
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3315
>             Project: Hadoop Map/Reduce
>          Issue Type: New Feature
>            Reporter: Sharad Agarwal
>            Assignee: Sharad Agarwal
>             Fix For: 0.24.0
>
>         Attachments: MAPREDUCE-3315-1.patch, MAPREDUCE-3315-2.patch, MAPREDUCE-3315.patch
>
>
> Currently master worker scenarios are forced fit into Map-Reduce. Now with YARN, these
can be first class and would benefit real/near realtime workloads and be more effective in
using the cluster resources.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message