hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Wangda Tan (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-8257) Native service should automatically adding escapes for environment/launch cmd before sending to YARN
Date Tue, 08 May 2018 00:15:00 GMT

    [ https://issues.apache.org/jira/browse/YARN-8257?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16466656#comment-16466656
] 

Wangda Tan commented on YARN-8257:
----------------------------------

Just took a closer look: 

Since both of environment/launch command will be written to a shell script and intercepted
by bash, we need to consider following chars should be escaped (add a \ before them)
{code:java}
` : execute a command
$ : reference to environment
\ : all other escapes
" : double quotes{code}
Reference: 

[https://superuser.com/questions/163515/bash-how-to-pass-command-line-arguments-containing-special-characters] (search
"per man bash")

> Native service should automatically adding escapes for environment/launch cmd before
sending to YARN
> ----------------------------------------------------------------------------------------------------
>
>                 Key: YARN-8257
>                 URL: https://issues.apache.org/jira/browse/YARN-8257
>             Project: Hadoop YARN
>          Issue Type: Bug
>          Components: yarn-native-services
>            Reporter: Wangda Tan
>            Assignee: Gour Saha
>            Priority: Critical
>
> Noticed this issue while using native service: 
> Basically, when a string for environment / launch command contains chars like ", /, `:
it needs to be escaped twice.
> The first time is from json spec, because of json accept double quote only, it needs
an escape.
> The second time is from launch container, what we did for command line is: (ContainerLaunch.java)
> {code:java}
> line("exec /bin/bash -c \"", StringUtils.join(" ", command), "\"");{code}
> And for environment:
> {code:java}
> line("export ", key, "=\"", value, "\"");{code}
> An example of launch_command: 
> {code:java}
> "launch_command": "export CLASSPATH=\\`\\$HADOOP_HDFS_HOME/bin/hadoop classpath --glob\\`"{code}
> And example of environment:
> {code:java}
> "TF_CONFIG" : "{\\\"cluster\\\": {\\\"master\\\": [\\\"master-0.distributed-tf.ambari-qa.tensorflow.site:8000\\\"],
\\\"ps\\\": [\\\"ps-0.distributed-tf.ambari-qa.tensorflow.site:8000\\\"], \\\"worker\\\":
[\\\"worker-0.distributed-tf.ambari-qa.tensorflow.site:8000\\\"]}, \\\"task\\\": {\\\"type\\\":\\\"${COMPONENT_NAME}\\\",
\\\"index\\\":${COMPONENT_ID}}, \\\"environment\\\":\\\"cloud\\\"}",{code}
> To improve usability, I think we should auto escape the input string once. (For example,
if user specified 
> {code}
> "TF_CONFIG": "\"key\""
> {code}
> We will automatically escape it to:
> {code}
> "TF_CONFIG": \\\"key\\\"
> {code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org


Mime
View raw message