aurora-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Josh Adams (JIRA)" <j...@apache.org>
Subject [jira] [Comment Edited] (AURORA-654) "Could not deserialize AssignedTask" from thermos_executor.pex
Date Wed, 20 Aug 2014 21:29:27 GMT

    [ https://issues.apache.org/jira/browse/AURORA-654?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14104572#comment-14104572
] 

Josh Adams edited comment on AURORA-654 at 8/20/14 9:27 PM:
------------------------------------------------------------

I think you were onto something regarding {{sys.path}}. Hopefully I discovered something to
support that suspicion just now. The old/working ttypes comes from the wheel in {{~/.pex/install}}

{noformat}
>>> import gen.apache.aurora.api.ttypes
>>> gen.apache.aurora.api.ttypes
<module 'gen.apache.aurora.api.ttypes' from '/foursquare/josh/.pex/install/src.main.thrift.org.apache.aurora.gen.py_thrift_c432d4bc-0.0.0-py2-none-any.whl.76707fc40c12ed46c8b6619914efd2cf14663ff4/src.main.thrift.org.apache.aurora.gen.py_thrift_c432d4bc-0.0.0-py2-none-any.whl/gen/apache/aurora/api/ttypes.py'>
>>>
{noformat}

Whereas the new one comes from within the generated pex's lib tree:

{noformat}
>>> import gen.apache.aurora.api.ttypes
>>> gen.apache.aurora.api.ttypes
<module 'gen.apache.aurora.api.ttypes' from '/foursquare/josh/loko/stagingjosh/infrastructure-aurora-base/installed/1aa579b/bin/aurora_client.pex/gen/apache/aurora/api/ttypes.pyc'>
>>>
{noformat}

Doing a {{diff}} on the {{ttypes.py}} from within the new {{aurora_client.pex}} and its respective
{{~/.pex/install/}} version of the Thrift wheel's {{ttypes.py}} shows a bunch of changes.
Here's the diff: https://gist.github.com/waxcorp/f17097f7d8f208430309

*edit* I've confirmed that the {{~/.pex/install}} version of {{ttypes.py}} installed by the
new {{aurora_client.pex}} works as before, so I believe the embedded {{gen/apache/aurora/api/ttypes.py*}}
files are to blame.

{noformat}
[josh@asgard api]$ pwd
/foursquare/josh/.pex/install/src.main.thrift.org.apache.aurora.gen.py_thrift_c432d4bc-0.0.0-py2-none-any.whl.76707fc40c12ed46c8b6619914efd2cf14663ff4/src.main.thrift.org.apache.aurora.gen.py_thrift_c432d4bc-0.0.0-py2-none-any.whl/gen/apache/aurora/api
[josh@asgard api]$ PYTHONPATH=/foursquare/josh/.pex/install/thrift-0.9.1-cp27-none-linux_x86_64.whl.5554a00a49bd3c9d0ed12679fbf9a9810c343ed1/thrift-0.9.1-cp27-none-linux_x86_64.whl/
PYTHONSTARTUP= python2.7
Python 2.7.6 (default, Apr  2 2014, 22:16:48)
[GCC 4.4.6 20120305 (Red Hat 4.4.6-4)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import ttypes
>>> ttypes
<module 'ttypes' from 'ttypes.pyc'>
>>> constraint = ttypes.Constraint()
>>> result = set()
>>> result.add(constraint)
>>> import thrift
>>> thrift
<module 'thrift' from '/foursquare/josh/.pex/install/thrift-0.9.1-cp27-none-linux_x86_64.whl.5554a00a49bd3c9d0ed12679fbf9a9810c343ed1/thrift-0.9.1-cp27-none-linux_x86_64.whl/thrift/__init__.pyc'>
>>>
{noformat}

Best,
Josh


was (Author: waxcorp):
I think you were onto something regarding {{sys.path}}. Hopefully I discovered something to
support that suspicion just now. The old/working ttypes comes from the wheel in {{~/.pex/install}}

{noformat}
>>> import gen.apache.aurora.api.ttypes
>>> gen.apache.aurora.api.ttypes
<module 'gen.apache.aurora.api.ttypes' from '/foursquare/josh/.pex/install/src.main.thrift.org.apache.aurora.gen.py_thrift_c432d4bc-0.0.0-py2-none-any.whl.76707fc40c12ed46c8b6619914efd2cf14663ff4/src.main.thrift.org.apache.aurora.gen.py_thrift_c432d4bc-0.0.0-py2-none-any.whl/gen/apache/aurora/api/ttypes.py'>
>>>
{noformat}

Whereas the new one comes from within the generated pex's lib tree:

{noformat}
>>> import gen.apache.aurora.api.ttypes
>>> gen.apache.aurora.api.ttypes
<module 'gen.apache.aurora.api.ttypes' from '/foursquare/josh/loko/stagingjosh/infrastructure-aurora-base/installed/1aa579b/bin/aurora_client.pex/gen/apache/aurora/api/ttypes.pyc'>
>>>
{noformat}

Doing a {{diff}} on the {{ttypes.py}} from within the new {{aurora_client.pex}} and its respective
{{~/.pex/install/}} version of the Thrift wheel's {{ttypes.py}} shows a bunch of changes.
Here's the diff: https://gist.github.com/waxcorp/f17097f7d8f208430309

*edit* I've confirmed that the {{~/.pex/install}} version of {{ttypes.py}} installed by the
new {{aurora_client.pex}} works as before, so I believe the embedded {{gen/apache/aurora/api/ttypes.py*}}
files are to blame.

{noformat}
[josh@asgard api]$ pwd
/foursquare/josh/.pex/install/src.main.thrift.org.apache.aurora.gen.py_thrift_c432d4bc-0.0.0-py2-none-any.whl.76707fc40c12ed46c8b6619914efd2cf14663ff4/src.main.thrift.org.apache.aurora.gen.py_thrift_c432d4bc-0.0.0-py2-none-any.whl/gen/apache/aurora/api
[josh@asgard api]$ PYTHONPATH=/foursquare/josh/.pex/install/thrift-0.9.1-cp27-none-linux_x86_64.whl.5554a00a49bd3c9d0ed12679fbf9a9810c343ed1/thrift-0.9.1-cp27-none-linux_x86_64.whl/
PYTHONSTARTUP= python2.7                                                                 
                                                                                         
                                        Python 2.7.6 (default, Apr  2 2014, 22:16:48)    
                                                                                         
                                                             [GCC 4.4.6 20120305 (Red Hat
4.4.6-4)] on linux2                                                                      
                                                                                  Type "help",
"copyright", "credits" or "license" for more information.
>>> import ttypes
>>> ttypes
<module 'ttypes' from 'ttypes.pyc'>
>>> constraint = ttypes.Constraint()
>>> result = set()
>>> result.add(constraint)
>>> import thrift
>>> thrift
<module 'thrift' from '/foursquare/josh/.pex/install/thrift-0.9.1-cp27-none-linux_x86_64.whl.5554a00a49bd3c9d0ed12679fbf9a9810c343ed1/thrift-0.9.1-cp27-none-linux_x86_64.whl/thrift/__init__.pyc'>
>>>
{noformat}

Best,
Josh

> "Could not deserialize AssignedTask" from thermos_executor.pex
> --------------------------------------------------------------
>
>                 Key: AURORA-654
>                 URL: https://issues.apache.org/jira/browse/AURORA-654
>             Project: Aurora
>          Issue Type: Bug
>          Components: Executor
>         Environment: CentOS 6.4 on x86_64 hardware
>            Reporter: Josh Adams
>
> We're seeing the following error when attempting to run a new task after upgrading to
incubator-aurora@8d98542 and mesos@dc0b7bf (0.19.1-rc1):
> {code}
> WARNING: Logging before InitGoogleLogging() is written to STDERR
> I0813 00:57:40.229567 10732 exec.cpp:131] Version: 0.19.1
> I0813 00:57:40.231796 10761 exec.cpp:205] Executor registered on slave
> 20140812-235601-84869386-5050-469-0
> FATAL] Could not deserialize AssignedTask
> FATAL] Traceback (most recent call last):
>   File "apache/aurora/executor/aurora_executor.py", line 234, in validate_task
>     assigned_task = assigned_task_from_mesos_task(task)
>   File "apache/aurora/executor/common/task_info.py", line 32, in
> assigned_task_from_mesos_task
>     assigned_task = thrift_deserialize(AssignedTask(), task.data)
>   File "/root/.pex/install/thrift-0.9.1-cp27-none-linux_x86_64.whl.5554a00a49bd3c9d0ed12679fbf9a9810c343ed1/thrift-0.9.1-cp27-none-linux_x86_64.whl/thrift/TSerialization.py",
> line 37, in deserialize
>     base.read(protocol)
>   File "gen/apache/aurora/api/ttypes.py", line 2841, in read
>     self.task.read(iprot)
>   File "gen/apache/aurora/api/ttypes.py", line 1678, in read
>     self.constraints.add(_elem26)
> TypeError: unhashable instance
> {code}
> We are upgrading to get the Announcer implementation working. Here's a simple task config
that has no {{constraints}} and still produces the error (we haven't been able to generate
a {{.aurora}} config that doesn't result in this exception.
> {code}
> drag_proc_0 = Process(
>   name = 'drag_proc_0',
>   cmdline = """
>     while sleep 1; do date; done
>   """
> )
> drag_task = Task(
>   processes = [drag_proc_0],
>   resources = Resources(cpu = 0.5, ram = 1*GB, disk = 1*GB)
> )
> jobs = [
>   Service(
>     task = drag_task,
>     cluster = 'staging',
>     role = 'produser',
>     environment = 'staging',
>     name = 'drag',
>     instances = 1,
>     update_config = UpdateConfig(batch_size = 1, watch_secs = 345, rollback_on_failure
= False, max_per_shard_failures = 3),
>   )
> ]
> {code}



--
This message was sent by Atlassian JIRA
(v6.2#6252)

Mime
View raw message