hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jason Lowe (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (MAPREDUCE-6369) MR compile shouldn't depend on nodemanager
Date Tue, 19 May 2015 18:23:07 GMT

    [ https://issues.apache.org/jira/browse/MAPREDUCE-6369?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14550944#comment-14550944
] 

Jason Lowe commented on MAPREDUCE-6369:
---------------------------------------

Ignoring dependencies due to unit tests, the dependency occurs in the ShuffleHandler which
runs inside the nodemanager as an auxiliary service.  The ShuffleHandler is looking for ContainerLocalizer.USERCACHE
and ContainerLocalizer.APPCACHE to help locate where the files are on disk.  These constants
could be moved to the yarn API package or we could add a mechanism for aux services to obtain
the application-specific local storage directories at runtime from the nodemanager.

> MR compile shouldn't depend on nodemanager
> ------------------------------------------
>
>                 Key: MAPREDUCE-6369
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-6369
>             Project: Hadoop Map/Reduce
>          Issue Type: Bug
>    Affects Versions: 2.7.0
>            Reporter: Karthik Kambatla
>
> Dependency analysis shows MR depends on nodemanager. If possible, MR shouldn't depend
on any yarn-server APIs at all. This might require some changes in Yarn. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message