hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Nikhil S. Ketkar (Updated) (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (MAPREDUCE-3315) Master-Worker Application on YARN
Date Wed, 04 Apr 2012 13:23:25 GMT

     [ https://issues.apache.org/jira/browse/MAPREDUCE-3315?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Nikhil S. Ketkar updated MAPREDUCE-3315:
----------------------------------------

    Attachment: MAPREDUCE-3315-1.patch

I have added a new patch which includes the previous work with one additional change regarding
the hostname of the Master (ApplicationMaster) which runs a Hadoop IPC server (Workers/Containers
are Hadoop IPC clients). Earlier, I had hardcoded the hostname. Now, the hostname of the Master
is identified and sent to each of the workers (containers) as an environment variable.

The port is still hard coded which is not good. What this means is that one physical node
can run only a single Master. I am thinking about specifying a list ports, if one is occupied
the next one in the list can be tried. I suppose it would be sufficient to have about 10-20.
Any suggessions? 


                
> Master-Worker Application on YARN
> ---------------------------------
>
>                 Key: MAPREDUCE-3315
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3315
>             Project: Hadoop Map/Reduce
>          Issue Type: New Feature
>            Reporter: Sharad Agarwal
>            Assignee: Sharad Agarwal
>             Fix For: 0.24.0
>
>         Attachments: MAPREDUCE-3315-1.patch, MAPREDUCE-3315.patch
>
>
> Currently master worker scenarios are forced fit into Map-Reduce. Now with YARN, these
can be first class and would benefit real/near realtime workloads and be more effective in
using the cluster resources.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message