hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Sangjin Lee (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (MAPREDUCE-6588) default values for PATH for tasks are not quite correct on Windows
Date Wed, 06 Jan 2016 00:19:39 GMT

     [ https://issues.apache.org/jira/browse/MAPREDUCE-6588?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Sangjin Lee updated MAPREDUCE-6588:
-----------------------------------
    Summary: default values for PATH for tasks are not quite correct on Windows  (was: default
values for PATH for tasks are not quite correct)

> default values for PATH for tasks are not quite correct on Windows
> ------------------------------------------------------------------
>
>                 Key: MAPREDUCE-6588
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-6588
>             Project: Hadoop Map/Reduce
>          Issue Type: Bug
>          Components: mr-am
>    Affects Versions: 2.6.0
>            Reporter: Sangjin Lee
>
> There are several issues with the way PATH are handled for MR tasks and AMs on Windows.
> - it does not get the PWD as part of the path, unlike the non-Windows case (see MAPREDUCE-6021)
> - the MR AM doesn't get the Hadoop's bin directory in the path explicitly through mapred
configuration; rather, it implicitly inherits it from the node manager (see MAPREDUCE-6577)
> - the default value for the MR task PATH uses "%...%" macros which may be expanded on
the client side prematurely; this is probably OK for tasks (as it's done by the AM), but not
OK for AMs; the late-binding cross-platform macros should be used
> I took a stab at addressing these as part of MAPREDUCE-6577, but I realized that it would
be a bigger issue than the fix for MAPREDUCE-6577 itself. Thus, I'm filing a separate JIRA
to address the Windows side of things.
> On a related note, there are quite a few brittle tests on testing these (TestMiniMRChildTask,
TestMiniMRClientCluster, etc.) which are built in such a way that the tests pass against the
current behavior but don't provide a lot of robust test value. We should also update those
tests to make them more robust.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message