aurora-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Joshua Cohen (JIRA)" <j...@apache.org>
Subject [jira] [Comment Edited] (AURORA-1782) Thermos Executor is not handling apostrophe gracefully
Date Wed, 05 Oct 2016 15:03:20 GMT

    [ https://issues.apache.org/jira/browse/AURORA-1782?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15549008#comment-15549008
] 

Joshua Cohen edited comment on AURORA-1782 at 10/5/16 3:02 PM:
---------------------------------------------------------------

I just tried to replicate this locally and I was unable to do so. I made the following change
to {{setup_env}} process in the [job used by our end to end tests|https://github.com/apache/aurora/blob/master/src/test/sh/org/apache/aurora/e2e/http/http_example.aurora#L45-L50]:

{noformat}
diff --git a/src/test/sh/org/apache/aurora/e2e/http/http_example.aurora b/src/test/sh/org/apache/aurora/e2e/http/http_example.aurora
index c71fb81..3b75948 100644
--- a/src/test/sh/org/apache/aurora/e2e/http/http_example.aurora
+++ b/src/test/sh/org/apache/aurora/e2e/http/http_example.aurora
@@ -45,7 +45,7 @@ stage_server = Process(
 setup_env = Process(
   name = 'setup_env',
   cmdline='''cat <<EOF > .thermos_profile
-export IT_WORKED=hello
+export IT_WORKED='hello'
 EOF'''
 )
{noformat}

And I was able to launch the job successfully (see attached screenshot). I'll need to dig
more to see what's causing your issue.




was (Author: joshua.cohen):
I just tried to replicate this locally and I was unable to do so. I made the following change
to {{setup_env}} process in the [job used by our end to end tests](https://github.com/apache/aurora/blob/master/src/test/sh/org/apache/aurora/e2e/http/http_example.aurora#L45-L50):

{noformat}
diff --git a/src/test/sh/org/apache/aurora/e2e/http/http_example.aurora b/src/test/sh/org/apache/aurora/e2e/http/http_example.aurora
index c71fb81..3b75948 100644
--- a/src/test/sh/org/apache/aurora/e2e/http/http_example.aurora
+++ b/src/test/sh/org/apache/aurora/e2e/http/http_example.aurora
@@ -45,7 +45,7 @@ stage_server = Process(
 setup_env = Process(
   name = 'setup_env',
   cmdline='''cat <<EOF > .thermos_profile
-export IT_WORKED=hello
+export IT_WORKED='hello'
 EOF'''
 )
{noformat}

And I was able to launch the job successfully (see attached screenshot). I'll need to dig
more to see what's causing your issue.



> Thermos Executor is not handling apostrophe gracefully
> ------------------------------------------------------
>
>                 Key: AURORA-1782
>                 URL: https://issues.apache.org/jira/browse/AURORA-1782
>             Project: Aurora
>          Issue Type: Bug
>    Affects Versions: 0.16.0
>            Reporter: Justin Venus
>         Attachments: process with single quotes.png
>
>
> This is a regression from the behavior in 0.15.0.
> This is enough to cause an execution to fail.
> {code}
> python -c 'import sys
> if __name__ == "__main__":
>   sys.exit(0)'
> {code}
> This is the error seen when running an inline script from a Process().
> {code}
> Failed to parse the flags: Failed to load flag 'command': Failed to load value '{"shell":true,"value":"/bin/bash
-c 'python -c import': syntax error at line 1 near: 
> {code}
> Ideally I could escape the apostrophe and have it work for Process's that run 'chrooted/pivoted'
by thermos or bare.  My work around has been to write a templating hack for aurora's dsl.
> {code}
> # Template.render(name='foo', template='SOME_STRING_TEMPLATE')
> # Returns a Process().  I can't just base64 the entire template (that would be too easy),

> # b/c then I can't use {{thermos.ports[http]}}.  So I go through and replace apostrophe
> # with a pattern that is unlikely to be legitimately used and render a 'cmdline' that
will 
> # undo the hack.  Of course using the octal or "'" directly proved troublesome, so I
had to 
> # base64 encode the simple sed statement and decode the script on the remote side.  Like
> # I said this is a complete hack.
> class Template(object):
>   class T(Struct):
>     payload = Required(String)
>   @classmethod
>   def render(cls, name=None, template=None, **kwargs):
>     assert name is not None
>     assert template is not None
>     return cls(name, template, 'template', **kwargs).process
>   def cmdline(self):
>     return ''.join([
>         '(mkdir .decoder && ',
>         '(test -x ./.decoder/decoder || ',
>         '((echo {{template_decoder}} | base64 -d > ./.decoder/decoder) &&
',
>         'chmod +x ./.decoder/decoder)) || ',
>         "(while [ ! -x ./.decoder/decoder ]; do echo resolving-decoder; sleep 1; done))
&& ",
>         '((echo "{{template_payload}}" | ./.decoder/decoder) > {{template_filename}})'
>     ])
>   def decoder_script(self):
>     return r"""#!/bin/bash
>     sed "s/__APOSTROPHE__/'/g"
>     """
>   def postInit(self):
>     self.process.in_scope = self.in_scope  # need to override imethod
>   def __init__(self, name, template, prefix, filename=None, auto_init=True, **kwargs):
>     self.resolved = False
>     self.process = Process(
>         name="%s:%s" % (prefix, name),
>         cmdline=self.cmdline(), **kwargs).bind(
>             self.__class__.T(payload=template.replace("'", '__APOSTROPHE__')),
>             template_filename=name if filename is None else filename,
>             template_decoder=base64.b64encode(self.decoder_script()))
>     if auto_init:
>       self.postInit()
>   def in_scope(self, *args, **kwargs):
>     """ensure name/commandline is resolved before proceeding"""
>     if self.resolved:
>       return self.process
>     self.process = Process.in_scope(self.process, *args, **kwargs)
>     if '{{' not in str(self.process.name()):
>       scopes = self.process.scopes()
>       for scope in scopes:
>         if not hasattr(scope, 'bind'):
>           continue
>         scope = scope.bind(**kwargs)
>         if scope.check() and isinstance(scope, self.__class__.T):
>           self.resolved = True
>           payload = str(scope.payload())
>           print(" INFO] Memoizing {}".format(self.process.name()))
>           self.process = self.process.bind(template_payload=payload)
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message