hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ahmed Radwan (Commented) (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (MAPREDUCE-3736) Variable substitution depth too large for fs.default.name causes jobs to fail
Date Fri, 03 Feb 2012 17:03:54 GMT

    [ https://issues.apache.org/jira/browse/MAPREDUCE-3736?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13199861#comment-13199861

Ahmed Radwan commented on MAPREDUCE-3736:

@Mahadev, The issue reported in this ticket is basically caused by the fact that: The "mapreduce.job.hdfs-servers"
in yarn-default.xml is set to ${fs.default.name}, and the sqoop build.xml sets <sysproperty
key="fs.default.name" value="${fs.default.name}"/> and this causes the indefinite circular
variable substitution.

The latest patch I uploaded uses fs.defaultFS instead of fs.default.name (also updating few
old places that were using fs.default.name) and also adds fs.default.name to core-default.xml
for backward compatibility. I have confirmed that the patch solves the sqoop failures.

@Eli, Are you also OK with the current shape of the patch.
> Variable substitution depth too large for fs.default.name causes jobs to fail
> -----------------------------------------------------------------------------
>                 Key: MAPREDUCE-3736
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3736
>             Project: Hadoop Map/Reduce
>          Issue Type: Bug
>          Components: mrv2
>    Affects Versions: 0.23.1
>            Reporter: Eli Collins
>            Assignee: Ahmed Radwan
>            Priority: Blocker
>         Attachments: MAPREDUCE-3736.patch, MAPREDUCE-3736_rev2.patch
> I'm seeing the same failure as MAPREDUCE-3462 in downstream projects running against
a recent build of branch-23. MR-3462 modified the tests rather than fixing the framework.
In that jira Ravi mentioned "I'm still ignorant of the change which made the tests start to
fail. I should probably understand better the reasons for that change before proposing a more
generalized fix." Let's figure out the general fix (rather than require all projects to set
mapreduce.job.hdfs-servers in their conf we should fix this in the framework). Perhaps we
should not default this config to "$fs.default.name"?

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira


View raw message