beam-builds mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Apache Jenkins Server <jenk...@builds.apache.org>
Subject Build failed in Jenkins: beam_PostCommit_Py_ValCont #7448
Date Tue, 16 Feb 2021 18:26:31 GMT
See <https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/7448/display/redirect?page=changes>

Changes:

[noreply] Move payload providers out into their own module. (#13963)


------------------------------------------
[...truncated 884.62 KB...]
copying apache_beam/tools/sideinput_microbenchmark.py -> apache-beam-2.29.0.dev0/apache_beam/tools
copying apache_beam/tools/utils.py -> apache-beam-2.29.0.dev0/apache_beam/tools
copying apache_beam/transforms/__init__.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/combinefn_lifecycle_pipeline.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/combinefn_lifecycle_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/combiners.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/combiners_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/core.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/create_source.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/create_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/cy_combiners.pxd -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/cy_combiners.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/cy_dataflow_distribution_counter.pxd -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/cy_dataflow_distribution_counter.pyx -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/dataflow_distribution_counter_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/deduplicate.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/deduplicate_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/display.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/display_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/dofn_lifecycle_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/environments.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/environments_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/external.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/external_it_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/external_java.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/external_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/external_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/external_test_py37.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/periodicsequence.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/periodicsequence_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/ptransform.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/ptransform_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/py_dataflow_distribution_counter.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/sideinputs.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/sideinputs_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/sql.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/sql_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/stats.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/stats_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/timeutil.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/transforms_keyword_only_args_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/trigger.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/trigger_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/userstate.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/userstate_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/util.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/util_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/validate_runner_xlang_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/window.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/window_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/write_ptransform_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/typehints/__init__.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/decorators.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/decorators_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/decorators_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/native_type_compatibility.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/native_type_compatibility_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/opcodes.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/row_type.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/schemas.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/schemas_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/sharded_key_type.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/sharded_key_type_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/trivial_inference.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/trivial_inference_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/trivial_inference_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typecheck.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typecheck_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typed_pipeline_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typed_pipeline_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typehints.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typehints_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typehints_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/utils/__init__.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/annotations.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/annotations_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/counters.pxd -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/counters.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/counters_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/histogram.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/histogram_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/interactive_utils.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/interactive_utils_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/plugin.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/processes.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/processes_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/profiler.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/profiler_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/proto_utils.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/retry.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/retry_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/sentinel.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/sharded_key.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/shared.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/shared_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/subprocess_server.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/subprocess_server_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/thread_pool_executor.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/thread_pool_executor_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/timestamp.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/timestamp_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/urns.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/windowed_value.pxd -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/windowed_value.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/windowed_value_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
Writing apache-beam-2.29.0.dev0/setup.cfg
creating dist
Creating tar archive
removing 'apache-beam-2.29.0.dev0' (and everything under it)
SDK_LOCATION=$(find dist/apache-beam-*.tar.gz)
find dist/apache-beam-*.tar.gz

# Run ValidatesRunner tests on Google Cloud Dataflow service
echo ">>> RUNNING DATAFLOW RUNNER VALIDATESCONTAINER TEST"
>>> RUNNING DATAFLOW RUNNER VALIDATESCONTAINER TEST
python setup.py nosetests \
  --attr ValidatesContainer \
  --nologcapture \
  --processes=1 \
  --process-timeout=900 \
  --with-xunitmp \
  --xunitmp-file=$XUNIT_FILE \
  --ignore-files '.*py3\d?\.py$' \
  --test-pipeline-options=" \
    --runner=TestDataflowRunner \
    --project=$PROJECT \
    --region=$REGION \
    --worker_harness_container_image=$CONTAINER:$TAG \
    --staging_location=$GCS_LOCATION/staging-validatesrunner-test \
    --temp_location=$GCS_LOCATION/temp-validatesrunner-test \
    --output=$GCS_LOCATION/output \
    --sdk_location=$SDK_LOCATION \
    --num_workers=1 \
    --prebuild_sdk_container_base_image=$CONTAINER:$TAG \
    --docker_registry_push_url=us.gcr.io/$PROJECT/$USER"
running nosetests
running egg_info
<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/container/venv/python3.6/lib/python3.6/site-packages/setuptools/dist.py>:461:
UserWarning: Normalizing '2.29.0.dev' to '2.29.0.dev0'
  warnings.warn(tmpl.format(**locals()))
<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/gen_protos.py>:243:
UserWarning: Installing grpcio-tools is recommended for development.
  warnings.warn('Installing grpcio-tools is recommended for development.')
INFO:gen_protos:Skipping proto regeneration: all files up to date
writing apache_beam.egg-info/PKG-INFO
writing dependency_links to apache_beam.egg-info/dependency_links.txt
writing entry points to apache_beam.egg-info/entry_points.txt
writing requirements to apache_beam.egg-info/requires.txt
writing top-level names to apache_beam.egg-info/top_level.txt
reading manifest file 'apache_beam.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
warning: no files found matching 'README.md'
warning: no files found matching 'NOTICE'
warning: no files found matching 'LICENSE'
writing manifest file 'apache_beam.egg-info/SOURCES.txt'
WARNING:root:python-snappy is not installed; some tests will be skipped.
WARNING:root:Tensorflow is not installed, so skipping some tests.
<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/io/vcfio.py>:50:
UserWarning: VCF IO will support Python 3 after migration to Nucleus, see: BEAM-5628.
  "VCF IO will support Python 3 after migration to Nucleus, "
WARNING:apache_beam.runners.interactive.interactive_environment:Dependencies required for
Interactive Beam PCollection visualization are not available, please use: `pip install apache-beam[interactive]`
to install necessary dependencies to enable all data visualization features.
WARNING:apache_beam.runners.interactive.interactive_environment:You cannot use Interactive
Beam features when you are not in an interactive environment such as a Jupyter notebook or
ipython terminal.
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.6 interpreter.
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.6 interpreter.
Sending build context to Docker daemon  1.736MB
Step 1/4 : FROM us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk:20210216-180033
 ---> af09834e7a30
Step 2/4 : RUN mkdir -p /opt/apache/beam/artifacts
 ---> Running in 43d0b412b93b
Removing intermediate container 43d0b412b93b
 ---> 038d5f329148
Step 3/4 : COPY ./* /opt/apache/beam/artifacts/
 ---> 92e0c8453c61
Step 4/4 : RUN /opt/apache/beam/boot --setup_only --artifacts /opt/apache/beam/artifacts/artifacts_info.json
 ---> Running in 270bd8ea247b
2021/02/16 18:21:22 Installing setup packages ...
2021/02/16 18:21:22 Failed to install required packages: failed to install SDK: package
'dataflow_python_sdk.tar' not found
The command '/bin/sh -c /opt/apache/beam/boot --setup_only --artifacts /opt/apache/beam/artifacts/artifacts_info.json'
returned a non-zero code: 1
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.6 interpreter.
test_wordcount_fnapi_it (apache_beam.examples.wordcount_it_test.WordCountIT) ... ok
test_wordcount_it_with_prebuilt_sdk_container (apache_beam.examples.wordcount_it_test.WordCountIT)
... ERROR
test_metrics_fnapi_it (apache_beam.runners.dataflow.dataflow_exercise_metrics_pipeline_test.ExerciseMetricsPipelineTest)
... ok

======================================================================
ERROR: test_wordcount_it_with_prebuilt_sdk_container (apache_beam.examples.wordcount_it_test.WordCountIT)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/portability/sdk_container_builder.py",>
line 173, in _invoke_docker_build_and_push
    cwd=self._temp_src_dir)
  File "/usr/lib/python3.6/subprocess.py", line 438, in run
    output=stdout, stderr=stderr)
subprocess.CalledProcessError: Command '['docker', 'build', '.', '-t', 'us.gcr.io/apache-beam-testing/jenkins/beam_python_prebuilt_sdk:6f6014c3-a229-4de5-80b9-0e0f974663ca']'
returned non-zero exit status 1.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/examples/wordcount_it_test.py",>
line 64, in test_wordcount_it_with_prebuilt_sdk_container
    prebuild_sdk_container_engine='local_docker')
  File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/examples/wordcount_it_test.py",>
line 107, in _run_wordcount_it
    save_main_session=False,
  File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/examples/wordcount.py",>
line 94, in run
    output | 'Write' >> WriteToText(known_args.output)
  File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/pipeline.py",>
line 580, in __exit__
    self.result = self.run()
  File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/pipeline.py",>
line 530, in run
    self._options).run(False)
  File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/pipeline.py",>
line 559, in run
    return self.runner.run_pipeline(self, self._options)
  File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/dataflow/test_dataflow_runner.py",>
line 57, in run_pipeline
    self).run_pipeline(pipeline, options)
  File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",>
line 497, in run_pipeline
    environments.DockerEnvironment.from_options(options))
  File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/transforms/environments.py",>
line 285, in from_options
    options)
  File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/portability/sdk_container_builder.py",>
line 140, in build_container_image
    return builder._build()
  File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/portability/sdk_container_builder.py",>
line 93, in _build
    self._invoke_docker_build_and_push(container_image_name)
  File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/portability/sdk_container_builder.py",>
line 177, in _invoke_docker_build_and_push
    'stderr:\n %s.' % err.stderr)
RuntimeError: Failed to build sdk container with local docker, stderr:
 None.

----------------------------------------------------------------------
XML: nosetests-beam_python3.6_sdk.xml
----------------------------------------------------------------------
XML: <https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/nosetests.xml>
----------------------------------------------------------------------
Ran 3 tests in 628.004s

FAILED (errors=1)
cleanup_container
Untagged: us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk:20210216-180033
Untagged: us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk@sha256:17f39a920dba42f3faa47770eaf047226eea53d20ea67b05f330f98a8d8f2e2f
docker images --format '{{.Repository}}:{{.Tag}}' | grep 'prebuilt_sdk'
"docker rmi" requires at least 1 argument.
See 'docker rmi --help'.

Usage:  docker rmi [OPTIONS] IMAGE [IMAGE...]

Remove one or more images
Failed to remove prebuilt sdk container
Digests:
- us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk@sha256:17f39a920dba42f3faa47770eaf047226eea53d20ea67b05f330f98a8d8f2e2f
  Associated tags:
 - 20210216-180033
Tags:
- us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk:20210216-180033
Deleted [us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk:20210216-180033].
Deleted [us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk@sha256:17f39a920dba42f3faa47770eaf047226eea53d20ea67b05f330f98a8d8f2e2f].
Removed the container
Build step 'Execute shell' marked build as failure

---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org


Mime
View raw message