hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Robert Kanter (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-7162) Remove XML excludes file format
Date Wed, 06 Sep 2017 18:20:00 GMT

    [ https://issues.apache.org/jira/browse/YARN-7162?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16155824#comment-16155824
] 

Robert Kanter commented on YARN-7162:
-------------------------------------

You can still pass in a timeout via the command line.  The disadvantage of getting rid of
the XML format (and also not having a JSON format) is that you can't have individually different
timeouts for different nodes.  In other words, {{yarn rmadmin refreshNodes -g 100 -server}}
will make all of the excluded nodes graceful decommission with a timeout of 100 - if you wanted
to have nodeA do 50, nodeB do 70, etc, you can't do that without the XML or JSON format. 
I'm not sure how useful that is.

The problem if we keep the XML format, is that
- if we want to later replace it with a JSON format, we can't simply remove the XML format
once we release it.  We'll have to deprecate it and wait for a later release (Hadoop 4?).
- if we want to keep it and have both XML and JSON, it would be good to keep the syntax similar
and matching with that in HDFS-9005, so we might want to change the XML format, which would
have a similar problem as the previous bullet.  We'd also probably want to rewrite the code
anyway to use a pluggable framework that [~mingma] was suggesting for YARN-5536.

And given the time constraints of Hadoop 3 and that it sounded like nobody would have time
to finish YARN-5536, it seemed like simply removing the XML format would let us get in front
of these issues because it hasn't actually been in a release yet.

> Remove XML excludes file format
> -------------------------------
>
>                 Key: YARN-7162
>                 URL: https://issues.apache.org/jira/browse/YARN-7162
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: graceful
>    Affects Versions: 2.9.0, 3.0.0-beta1
>            Reporter: Robert Kanter
>            Assignee: Robert Kanter
>            Priority: Blocker
>
> YARN-5536 aims to replace the XML format for the excludes file with a JSON format.  However,
it looks like we won't have time for that for Hadoop 3 Beta 1.  The concern is that if we
release it as-is, we'll now have to support the XML format as-is for all of Hadoop 3.x, which
we're either planning on removing, or rewriting using a pluggable framework.  
> [This comment in YARN-5536|https://issues.apache.org/jira/browse/YARN-5536?focusedCommentId=16126194&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-16126194]
proposed two quick solutions to prevent this compat issue.  In this JIRA, we're going to remove
the XML format.  If we later want to add it back in, YARN-5536 can add it back, rewriting
it to be in the pluggable framework.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org


Mime
View raw message