beam-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Apache Jenkins Server <jenk...@builds.apache.org>
Subject Build failed in Jenkins: beam_PostRelease_NightlySnapshot #42
Date Tue, 13 Feb 2018 00:07:36 GMT
See <https://builds.apache.org/job/beam_PostRelease_NightlySnapshot/42/display/redirect>

------------------------------------------
[...truncated 509.95 KB...]
copying apache_beam/utils/retry_test.py -> apache-beam-2.3.0/apache_beam/utils
copying apache_beam/utils/timestamp.py -> apache-beam-2.3.0/apache_beam/utils
copying apache_beam/utils/timestamp_test.py -> apache-beam-2.3.0/apache_beam/utils
copying apache_beam/utils/urns.py -> apache-beam-2.3.0/apache_beam/utils
copying apache_beam/utils/windowed_value.pxd -> apache-beam-2.3.0/apache_beam/utils
copying apache_beam/utils/windowed_value.py -> apache-beam-2.3.0/apache_beam/utils
copying apache_beam/utils/windowed_value_test.py -> apache-beam-2.3.0/apache_beam/utils
Writing apache-beam-2.3.0/setup.cfg
creating dist
Creating tar archive
removing 'apache-beam-2.3.0' (and everything under it)
Processing ./dist/apache-beam-2.3.0.tar.gz
Collecting avro<2.0.0,>=1.8.1 (from apache-beam==2.3.0)
/tmp/groovy-generated-7662706818481476792-tmpdir/apache-beam-2.3.0/sdks/python/temp_virtualenv/local/lib/python2.7/site-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py:318:
SNIMissingWarning: An HTTPS request has been made, but the SNI (Subject Name Indication) extension
to TLS is not available on this platform. This may cause the server to present an incorrect
TLS certificate, which can cause validation failures. You can upgrade to a newer version of
Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/security.html#snimissingwarning.
  SNIMissingWarning
/tmp/groovy-generated-7662706818481476792-tmpdir/apache-beam-2.3.0/sdks/python/temp_virtualenv/local/lib/python2.7/site-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py:122:
InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3
from configuring SSL appropriately and may cause certain SSL connections to fail. You can
upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/security.html#insecureplatformwarning.
  InsecurePlatformWarning
Collecting crcmod<2.0,>=1.7 (from apache-beam==2.3.0)
Collecting dill==0.2.6 (from apache-beam==2.3.0)
Collecting grpcio<2,>=1.0 (from apache-beam==2.3.0)
  Using cached grpcio-1.9.1-cp27-cp27mu-manylinux1_x86_64.whl
Collecting httplib2<0.10,>=0.8 (from apache-beam==2.3.0)
Collecting mock<3.0.0,>=1.0.1 (from apache-beam==2.3.0)
  Using cached mock-2.0.0-py2.py3-none-any.whl
Collecting oauth2client<5,>=2.0.1 (from apache-beam==2.3.0)
  Using cached oauth2client-4.1.2-py2.py3-none-any.whl
Collecting protobuf<4,>=3.5.0.post1 (from apache-beam==2.3.0)
  Using cached protobuf-3.5.1-cp27-cp27mu-manylinux1_x86_64.whl
Collecting pyyaml<4.0.0,>=3.12 (from apache-beam==2.3.0)
Collecting pyvcf<0.7.0,>=0.6.8 (from apache-beam==2.3.0)
Collecting six<1.12,>=1.9 (from apache-beam==2.3.0)
  Using cached six-1.11.0-py2.py3-none-any.whl
Collecting typing<3.7.0,>=3.6.0 (from apache-beam==2.3.0)
  Using cached typing-3.6.4-py2-none-any.whl
Collecting futures<4.0.0,>=3.1.1 (from apache-beam==2.3.0)
  Using cached futures-3.2.0-py2-none-any.whl
Collecting hdfs3<0.4.0,>=0.3.0 (from apache-beam==2.3.0)
  Using cached hdfs3-0.3.0-py2.py3-none-any.whl
Collecting google-apitools<=0.5.20,>=0.5.18 (from apache-beam==2.3.0)
  Using cached google_apitools-0.5.20-py2-none-any.whl
Collecting proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0 (from apache-beam==2.3.0)
Collecting googledatastore==7.0.1 (from apache-beam==2.3.0)
Collecting google-cloud-pubsub==0.26.0 (from apache-beam==2.3.0)
  Using cached google_cloud_pubsub-0.26.0-py2.py3-none-any.whl
Collecting google-cloud-bigquery==0.25.0 (from apache-beam==2.3.0)
  Using cached google_cloud_bigquery-0.25.0-py2.py3-none-any.whl
Collecting enum34>=1.0.4 (from grpcio<2,>=1.0->apache-beam==2.3.0)
  Using cached enum34-1.1.6-py2-none-any.whl
Collecting funcsigs>=1; python_version < "3.3" (from mock<3.0.0,>=1.0.1->apache-beam==2.3.0)
  Using cached funcsigs-1.0.2-py2.py3-none-any.whl
Collecting pbr>=0.11 (from mock<3.0.0,>=1.0.1->apache-beam==2.3.0)
  Using cached pbr-3.1.1-py2.py3-none-any.whl
Collecting rsa>=3.1.4 (from oauth2client<5,>=2.0.1->apache-beam==2.3.0)
  Using cached rsa-3.4.2-py2.py3-none-any.whl
Collecting pyasn1-modules>=0.0.5 (from oauth2client<5,>=2.0.1->apache-beam==2.3.0)
  Using cached pyasn1_modules-0.2.1-py2.py3-none-any.whl
Collecting pyasn1>=0.1.7 (from oauth2client<5,>=2.0.1->apache-beam==2.3.0)
  Using cached pyasn1-0.4.2-py2.py3-none-any.whl
Requirement already satisfied: setuptools in ./temp_virtualenv/lib/python2.7/site-packages
(from protobuf<4,>=3.5.0.post1->apache-beam==2.3.0)
Collecting fasteners>=0.14 (from google-apitools<=0.5.20,>=0.5.18->apache-beam==2.3.0)
  Using cached fasteners-0.14.1-py2.py3-none-any.whl
Collecting googleapis-common-protos<2.0dev,>=1.5.2 (from proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0->apache-beam==2.3.0)
Collecting google-cloud-core<0.26dev,>=0.25.0 (from google-cloud-pubsub==0.26.0->apache-beam==2.3.0)
  Using cached google_cloud_core-0.25.0-py2.py3-none-any.whl
Collecting gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0 (from google-cloud-pubsub==0.26.0->apache-beam==2.3.0)
Collecting monotonic>=0.1 (from fasteners>=0.14->google-apitools<=0.5.20,>=0.5.18->apache-beam==2.3.0)
  Using cached monotonic-1.4-py2.py3-none-any.whl
Collecting google-auth<2.0.0dev,>=0.4.0 (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0)
  Using cached google_auth-1.4.1-py2.py3-none-any.whl
Collecting google-auth-httplib2 (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0)
  Using cached google_auth_httplib2-0.0.3-py2.py3-none-any.whl
Collecting google-gax<0.16dev,>=0.15.7 (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0)
  Using cached google_gax-0.15.16-py2.py3-none-any.whl
Collecting grpc-google-iam-v1<0.12dev,>=0.11.1 (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0)
Collecting proto-google-cloud-pubsub-v1[grpc]<0.16dev,>=0.15.4 (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0)
Collecting cachetools>=2.0.0 (from google-auth<2.0.0dev,>=0.4.0->google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0)
  Using cached cachetools-2.0.1-py2.py3-none-any.whl
Collecting future<0.17dev,>=0.16.0 (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0)
Collecting ply==3.8 (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0)
Collecting requests<3.0dev,>=2.13.0 (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0)
  Using cached requests-2.18.4-py2.py3-none-any.whl
Collecting urllib3<1.23,>=1.21.1 (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0)
  Using cached urllib3-1.22-py2.py3-none-any.whl
Collecting idna<2.7,>=2.5 (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0)
  Using cached idna-2.6-py2.py3-none-any.whl
Collecting chardet<3.1.0,>=3.0.2 (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0)
  Using cached chardet-3.0.4-py2.py3-none-any.whl
Collecting certifi>=2017.4.17 (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0)
  Using cached certifi-2018.1.18-py2.py3-none-any.whl
Building wheels for collected packages: apache-beam
  Running setup.py bdist_wheel for apache-beam: started
  Running setup.py bdist_wheel for apache-beam: finished with status 'done'
  Stored in directory: /home/jenkins/.cache/pip/wheels/20/3b/4d/43d3bcfd7cf9a35f7a9acce994b3b5aa9a3d86d615291ae7f4
Successfully built apache-beam
Installing collected packages: avro, crcmod, dill, futures, six, protobuf, enum34, grpcio,
httplib2, funcsigs, pbr, mock, pyasn1, rsa, pyasn1-modules, oauth2client, pyyaml, pyvcf, typing,
hdfs3, monotonic, fasteners, google-apitools, googleapis-common-protos, proto-google-cloud-datastore-v1,
googledatastore, cachetools, google-auth, google-auth-httplib2, google-cloud-core, future,
ply, urllib3, idna, chardet, certifi, requests, google-gax, grpc-google-iam-v1, proto-google-cloud-pubsub-v1,
gapic-google-cloud-pubsub-v1, google-cloud-pubsub, google-cloud-bigquery, apache-beam
Successfully installed apache-beam-2.3.0 avro-1.8.2 cachetools-2.0.1 certifi-2018.1.18 chardet-3.0.4
crcmod-1.7 dill-0.2.6 enum34-1.1.6 fasteners-0.14.1 funcsigs-1.0.2 future-0.16.0 futures-3.2.0
gapic-google-cloud-pubsub-v1-0.15.4 google-apitools-0.5.20 google-auth-1.4.1 google-auth-httplib2-0.0.3
google-cloud-bigquery-0.25.0 google-cloud-core-0.25.0 google-cloud-pubsub-0.26.0 google-gax-0.15.16
googleapis-common-protos-1.5.3 googledatastore-7.0.1 grpc-google-iam-v1-0.11.4 grpcio-1.9.1
hdfs3-0.3.0 httplib2-0.9.2 idna-2.6 mock-2.0.0 monotonic-1.4 oauth2client-4.1.2 pbr-3.1.1
ply-3.8 proto-google-cloud-datastore-v1-0.90.4 proto-google-cloud-pubsub-v1-0.15.4 protobuf-3.5.1
pyasn1-0.4.2 pyasn1-modules-0.2.1 pyvcf-0.6.8 pyyaml-3.12 requests-2.18.4 rsa-3.4.2 six-1.11.0
typing-3.6.4 urllib3-1.22

----------------------------------------------------------------
Running wordcount example with DirectRunner and verify results
----------------------------------------------------------------
python -m apache_beam.examples.wordcount --output wordcount_direct.txt
INFO:root:Missing pipeline option (runner). Executing pipeline using the default runner: DirectRunner.
apache_beam/io/gcp/gcsio.py:160: DeprecationWarning: object() takes no parameters
  super(GcsIO, cls).__new__(cls, storage_client))
INFO:root:Starting the size estimation of the input
INFO:oauth2client.transport:Attempting refresh to obtain initial access_token
INFO:root:Finished the size estimation of the input at 1 files. Estimation took 0.0828478336334
seconds
apache_beam/coders/typecoders.py:133: UserWarning: Using fallback coder for typehint: <type
'NoneType'>.
  warnings.warn('Using fallback coder for typehint: %r.' % typehint)
apache_beam/coders/typecoders.py:133: UserWarning: Using fallback coder for typehint: Any.
  warnings.warn('Using fallback coder for typehint: %r.' % typehint)
INFO:root:Running pipeline with DirectRunner.
INFO:root:Starting the size estimation of the input
INFO:oauth2client.transport:Attempting refresh to obtain initial access_token
INFO:root:Finished the size estimation of the input at 1 files. Estimation took 0.0766088962555
seconds
INFO:oauth2client.transport:Attempting refresh to obtain initial access_token
INFO:root:Starting finalize_write threads with num_shards: 1, batches: 1, num_threads: 1
INFO:root:Renamed 1 shards in 0.12 seconds.
INFO:root:number of empty lines: 1663
INFO:root:average word length: 4
ls
apache_beam
apache_beam.egg-info
build.gradle
container
dist
findSupportedPython.groovy
generate_pydoc.sh
gen_protos.py
gen_protos.pyc
MANIFEST.in
pom.xml
run_postcommit.sh
run_pylint.sh
run_validatesrunner.sh
setup.cfg
setup.py
temp_virtualenv
test_config.py
tox.ini
wordcount_direct.txt-00000-of-00001
Verified wordcount_direct.txt-00000-of-00001

----------------------------------------------------------------
Running wordcount example with DataflowRunner with command: 
python -m apache_beam.examples.wordcount --output gs://yifan_auto_verification_test_bucket/wordcount_direct.txt
--staging_location gs://yifan_auto_verification_test_bucket/temp --temp_location gs://yifan_auto_verification_test_bucket/temp
--runner DataflowRunner --job_name wordcount --project my-first-project-190318 --num_workers
1 --sdk_location dist/apache-beam-2.3.0.tar.gz 
----------------------------------------------------------------
python -m apache_beam.examples.wordcount --output gs://yifan_auto_verification_test_bucket/wordcount_direct.txt
--staging_location gs://yifan_auto_verification_test_bucket/temp --temp_location gs://yifan_auto_verification_test_bucket/temp
--runner DataflowRunner --job_name wordcount --project my-first-project-190318 --num_workers
1 --sdk_location dist/apache-beam-2.3.0.tar.gz 
apache_beam/io/gcp/gcsio.py:160: DeprecationWarning: object() takes no parameters
  super(GcsIO, cls).__new__(cls, storage_client))
INFO:root:Starting the size estimation of the input
INFO:oauth2client.transport:Attempting refresh to obtain initial access_token
INFO:root:Finished the size estimation of the input at 1 files. Estimation took 0.0796480178833
seconds
apache_beam/coders/typecoders.py:133: UserWarning: Using fallback coder for typehint: <type
'NoneType'>.
  warnings.warn('Using fallback coder for typehint: %r.' % typehint)
apache_beam/coders/typecoders.py:133: UserWarning: Using fallback coder for typehint: Any.
  warnings.warn('Using fallback coder for typehint: %r.' % typehint)
INFO:root:Starting the size estimation of the input
INFO:oauth2client.transport:Attempting refresh to obtain initial access_token
INFO:root:Finished the size estimation of the input at 1 files. Estimation took 0.0779528617859
seconds
INFO:root:Starting GCS upload to gs://yifan_auto_verification_test_bucket/temp/wordcount.1518480439.014223/pipeline.pb...
INFO:oauth2client.transport:Attempting refresh to obtain initial access_token
Traceback (most recent call last):
  File "/usr/lib/python2.7/runpy.py", line 162, in _run_module_as_main
    "__main__", fname, loader, pkg_name)
  File "/usr/lib/python2.7/runpy.py", line 72, in _run_code
    exec code in run_globals
  File "/tmp/groovy-generated-7662706818481476792-tmpdir/apache-beam-2.3.0/sdks/python/apache_beam/examples/wordcount.py",
line 134, in <module>
    run()
  File "/tmp/groovy-generated-7662706818481476792-tmpdir/apache-beam-2.3.0/sdks/python/apache_beam/examples/wordcount.py",
line 113, in run
    result = p.run()
  File "apache_beam/pipeline.py", line 333, in run
    self.to_runner_api(), self.runner, self._options).run(False)
  File "apache_beam/pipeline.py", line 342, in run
    return self.runner.run_pipeline(self)
  File "apache_beam/runners/dataflow/dataflow_runner.py", line 315, in run_pipeline
    self.dataflow_client.create_job(self.job), self)
  File "apache_beam/utils/retry.py", line 175, in wrapper
    return fun(*args, **kwargs)
  File "apache_beam/runners/dataflow/internal/apiclient.py", line 461, in create_job
    self.create_job_description(job)
  File "apache_beam/runners/dataflow/internal/apiclient.py", line 487, in create_job_description
    StringIO(job.proto_pipeline.SerializeToString()))
  File "apache_beam/runners/dataflow/internal/apiclient.py", line 453, in stage_file
    (gcs_or_local_path, reportable_errors[e.status_code]))
IOError: Could not upload to GCS path gs://yifan_auto_verification_test_bucket/temp/wordcount.1518480439.014223:
access denied. Please verify that credentials are valid and that you have write access to
the specified path.
apache_beam/io/gcp/gcsio.py:160: DeprecationWarning: object() takes no parameters
  super(GcsIO, cls).__new__(cls, storage_client))
INFO:root:Starting the size estimation of the input
INFO:oauth2client.transport:Attempting refresh to obtain initial access_token
INFO:root:Finished the size estimation of the input at 1 files. Estimation took 0.0796480178833
seconds
apache_beam/coders/typecoders.py:133: UserWarning: Using fallback coder for typehint: <type
'NoneType'>.
  warnings.warn('Using fallback coder for typehint: %r.' % typehint)
apache_beam/coders/typecoders.py:133: UserWarning: Using fallback coder for typehint: Any.
  warnings.warn('Using fallback coder for typehint: %r.' % typehint)
INFO:root:Starting the size estimation of the input
INFO:oauth2client.transport:Attempting refresh to obtain initial access_token
INFO:root:Finished the size estimation of the input at 1 files. Estimation took 0.0779528617859
seconds
INFO:root:Starting GCS upload to gs://yifan_auto_verification_test_bucket/temp/wordcount.1518480439.014223/pipeline.pb...
INFO:oauth2client.transport:Attempting refresh to obtain initial access_token
Traceback (most recent call last):
  File "/usr/lib/python2.7/runpy.py", line 162, in _run_module_as_main
    "__main__", fname, loader, pkg_name)
  File "/usr/lib/python2.7/runpy.py", line 72, in _run_code
    exec code in run_globals
  File "/tmp/groovy-generated-7662706818481476792-tmpdir/apache-beam-2.3.0/sdks/python/apache_beam/examples/wordcount.py",
line 134, in <module>
    run()
  File "/tmp/groovy-generated-7662706818481476792-tmpdir/apache-beam-2.3.0/sdks/python/apache_beam/examples/wordcount.py",
line 113, in run
    result = p.run()
  File "apache_beam/pipeline.py", line 333, in run
    self.to_runner_api(), self.runner, self._options).run(False)
  File "apache_beam/pipeline.py", line 342, in run
    return self.runner.run_pipeline(self)
  File "apache_beam/runners/dataflow/dataflow_runner.py", line 315, in run_pipeline
    self.dataflow_client.create_job(self.job), self)
  File "apache_beam/utils/retry.py", line 175, in wrapper
    return fun(*args, **kwargs)
  File "apache_beam/runners/dataflow/internal/apiclient.py", line 461, in create_job
    self.create_job_description(job)
  File "apache_beam/runners/dataflow/internal/apiclient.py", line 487, in create_job_description
    StringIO(job.proto_pipeline.SerializeToString()))
  File "apache_beam/runners/dataflow/internal/apiclient.py", line 453, in stage_file
    (gcs_or_local_path, reportable_errors[e.status_code]))
IOError: Could not upload to GCS path gs://yifan_auto_verification_test_bucket/temp/wordcount.1518480439.014223:
access denied. Please verify that credentials are valid and that you have write access to
the specified path.
[ERROR] Failed command: 

:release:runQuickstartsPython FAILED

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':release:runQuickstartsPython'.
> Process 'command '/usr/local/asfpackages/java/jdk1.8.0_152/bin/java'' finished with non-zero
exit value 1

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to
get more log output. Run with --scan to get full insights.

* Get more help at https://help.gradle.org

BUILD FAILED in 1m 32s
2 actionable tasks: 2 executed
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
Not sending mail to unregistered user ccy@google.com
Not sending mail to unregistered user daniel.o.programmer@gmail.com
Not sending mail to unregistered user jb@nanthrax.net
Not sending mail to unregistered user apilloud@google.com
Not sending mail to unregistered user fjetumale@gmail.com
Not sending mail to unregistered user github@alasdairhodge.co.uk

Mime
View raw message