hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Shane Kumpf (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-6830) Support quoted strings for environment variables
Date Thu, 20 Jul 2017 15:42:00 GMT

    [ https://issues.apache.org/jira/browse/YARN-6830?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16094847#comment-16094847

Shane Kumpf commented on YARN-6830:

Thanks for the feedback [~templedf]. I see what you are proposing and it's close, but unfortunately,
that still seems to break down when there are embedded commas. I'm all for simplifying this,
but after quite a bit of testing, I'm still not able to come up with a single regex that works
for all cases, given the features that java supports. I'm certainly open to additional suggestions.

With what you've proposed above, the result is:

    String goodEnv = "a1='1',b_2=2,_c=3,d=4,e=,f_win=%JAVA_HOME%"
        + ",g_nix=$JAVA_HOME,h='6,7',i=\"8,9\",j=\"10 11\"";

Key: a1 Value: '1'
Key: b_2 Value: 2
Key: _c Value: 3
Key: d Value: 4
Key: e Value: 
Key: f_win Value: %JAVA_HOME%
Key: g_nix Value: $JAVA_HOME
Key: h Value: '6
Key: i Value: "8
Key: j Value: "10 11"

> Support quoted strings for environment variables
> ------------------------------------------------
>                 Key: YARN-6830
>                 URL: https://issues.apache.org/jira/browse/YARN-6830
>             Project: Hadoop YARN
>          Issue Type: Bug
>            Reporter: Shane Kumpf
>            Assignee: Shane Kumpf
>         Attachments: YARN-6830.001.patch
> There are cases where it is necessary to allow for quoted string literals within environment
variables values when passed via the yarn command line interface.
> For example, consider the follow environment variables for a MR map task.
> {{MODE=bar}}
> {{IMAGE_NAME=foo}}
> {{MOUNTS=/tmp/foo,/tmp/bar}}
> When running the MR job, these environment variables are supplied as a comma delimited
> {{-Dmapreduce.map.env="MODE=bar,IMAGE_NAME=foo,MOUNTS=/tmp/foo,/tmp/bar"}}
> In this case, {{MOUNTS}} will be parsed and added to the task environment as {{MOUNTS=/tmp/foo}}.
Any attempts to quote the embedded comma separated value results in quote characters becoming
part of the value, and parsing still breaks down at the comma.
> This issue is to allow for quoting the comma separated value (escaped double or single
quote). This was mentioned on YARN-4595 and will impact YARN-5534 as well.

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org

View raw message