beam-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Pablo Estrada (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (BEAM-4105) Python Postcommit suite does not start on Jenkins - proto generation failed
Date Tue, 17 Apr 2018 23:50:00 GMT

    [ https://issues.apache.org/jira/browse/BEAM-4105?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16441678#comment-16441678
] 

Pablo Estrada commented on BEAM-4105:
-------------------------------------

I am not 100% sure, but I think this happens because pip install -e .[gcp,test] installs dependencies
from within /tmp directories, and the gen_protos call expects the directory structure to be
like so:
{code:java}
beam/
-model/
-sdks/
--python/
{code}

In https://github.com/apache/beam/pull/5155/files I'm trying to run python setup.py install
before running pip install, so that gen_protos will run before the pip install. I need to
refactor the code a bit to be able to run this and change to a gradle build gradually.

> Python Postcommit suite does not start on Jenkins - proto generation failed
> ---------------------------------------------------------------------------
>
>                 Key: BEAM-4105
>                 URL: https://issues.apache.org/jira/browse/BEAM-4105
>             Project: Beam
>          Issue Type: Improvement
>          Components: testing
>            Reporter: Valentyn Tymofieiev
>            Assignee: Jason Kuster
>            Priority: Major
>
> The suite has been steadily failing for last 3 days. Sample logs:  
> # Tox runs unit tests in a virtual environment
> ${LOCAL_PATH}/tox -e ALL -c sdks/python/tox.ini
> GLOB sdist-make: /home/jenkins/jenkins-slave/workspace/beam_PostCommit_Python_Verify/src/sdks/python/setup.py
> ERROR: invocation failed (exit code 1), logfile: /home/jenkins/jenkins-slave/workspace/beam_PostCommit_Python_Verify/src/sdks/python/target/.tox/log/tox-0.log
> ERROR: actionid: tox
> msg: packaging
> cmdargs: ['/usr/bin/python', local('/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Python_Verify/src/sdks/python/setup.py'),
'sdist', '--formats=zip', '--dist-dir', local('/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Python_Verify/src/sdks/python/target/.tox/dist')]
> /usr/local/lib/python2.7/dist-packages/setuptools/dist.py:397: UserWarning: Normalizing
'2.5.0.dev' to '2.5.0.dev0'
>   normalized_version,
> Regenerating common_urns module.
> running sdist
> /home/jenkins/jenkins-slave/workspace/beam_PostCommit_Python_Verify/src/sdks/python/gen_protos.py:51:
UserWarning: Installing grpcio-tools is recommended for development.
>   warnings.warn('Installing grpcio-tools is recommended for development.')
> WARNING:root:Installing grpcio-tools into /home/jenkins/jenkins-slave/workspace/beam_PostCommit_Python_Verify/src/sdks/python/.eggs/grpcio-wheels
> Process Process-1:
> Traceback (most recent call last):
>   File "/usr/lib/python2.7/multiprocessing/process.py", line 258, in _bootstrap
>     self.run()
>   File "/usr/lib/python2.7/multiprocessing/process.py", line 114, in run
>     self._target(*self._args, **self._kwargs)
>   File "/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Python_Verify/src/sdks/python/gen_protos.py",
line 149, in _install_grpcio_tools_and_generate_proto_files
>     shutil.rmtree(build_path)
>   File "/usr/lib/python2.7/shutil.py", line 239, in rmtree
>     onerror(os.listdir, path, sys.exc_info())
>   File "/usr/lib/python2.7/shutil.py", line 237, in rmtree
>     names = os.listdir(path)
> OSError: [Errno 2] No such file or directory: '/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Python_Verify/src/sdks/python/.eggs/grpcio-wheels-build'
> Traceback (most recent call last):
>   File "setup.py", line 235, in <module>
>     'test': generate_protos_first(test),
>   File "/usr/local/lib/python2.7/dist-packages/setuptools/__init__.py", line 129, in
setup
>     return distutils.core.setup(**attrs)
>   File "/usr/lib/python2.7/distutils/core.py", line 151, in setup
>     dist.run_commands()
>   File "/usr/lib/python2.7/distutils/dist.py", line 953, in run_commands
>     self.run_command(cmd)
>   File "/usr/lib/python2.7/distutils/dist.py", line 972, in run_command
>     cmd_obj.run()
>   File "setup.py", line 141, in run
>     gen_protos.generate_proto_files()
>   File "/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Python_Verify/src/sdks/python/gen_protos.py",
line 97, in generate_proto_files
>     raise ValueError("Proto generation failed (see log for details).")
> ValueError: Proto generation failed (see log for details).
> ERROR: FAIL could not package project - v = InvocationError('/usr/bin/python /home/jenkins/jenkins-slave/workspace/beam_PostCommit_Python_Verify/src/sdks/python/setup.py
sdist --formats=zip --dist-dir /home/jenkins/jenkins-slave/workspace/beam_PostCommit_Python_Verify/src/sdks/python/target/.tox/dist
(see /home/jenkins/jenkins-slave/workspace/beam_PostCommit_Python_Verify/src/sdks/python/target/.tox/log/tox-0.log)',
1)
> Build step 'Execute shell' marked build as failure
> Sending e-mails to: commits@beam.apache.org
> Finished: FAILURE



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Mime
View raw message