hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jason Lowe (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-3403) Nodemanager dies after a small typo in mapred-site.xml is induced
Date Wed, 02 Sep 2015 13:17:46 GMT

    [ https://issues.apache.org/jira/browse/YARN-3403?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14727315#comment-14727315
] 

Jason Lowe commented on YARN-3403:
----------------------------------

I think the real issue here is the nodemanager is not supposed to be constantly re-reading
the configuration files.  Normally an explicit refresh admin command or restart of the daemon
is expected to be performed before daemons recognize updated configs.  The problem seems to
stem from the fact that BCFile.WBlockState is creating a compressor without providing the
configuration.  When the codec pool gets invoked without a conf it ends up creating its own
which causes the confs to be read from disk each time.

> Nodemanager dies after a small typo in mapred-site.xml is induced
> -----------------------------------------------------------------
>
>                 Key: YARN-3403
>                 URL: https://issues.apache.org/jira/browse/YARN-3403
>             Project: Hadoop YARN
>          Issue Type: Bug
>            Reporter: Nikhil Mulley
>            Priority: Critical
>
> Hi,
> We have noticed that with a small typo in terms of xml config (mapred-site.xml) can cause
the nodemanager go down completely without stopping/restarting it externally.
> I find it little weird that editing the config files on the filesystem, could cause the
running slave daemon yarn nodemanager shutdown.
> In this case, I had a ending tag '/' missed in a property and that induced the nodemanager
go down in a cluster. 
> Why would nodemanager reload the configs while it is running? Are not they picked up
when they are started? Even if they are automated to pick up the new configs dynamically,
I think the xmllint/config checker should come in before the nodemanager is asked to reload/restart.
>  
> ---
> java.lang.RuntimeException: org.xml.sax.SAXParseException; systemId: file:/etc/hadoop/conf/mapred-site.xml;
lineNumber: 228; columnNumber: 3; The element type "value" must be terminated by the matching
end-tag "</value>".
>        at org.apache.hadoop.conf.Configuration.loadResource(Configuration.java:2348)
> ---
> Please shed light on this.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message