beam-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Apache Jenkins Server <jenk...@builds.apache.org>
Subject Build failed in Jenkins: beam_PerformanceTests_HadoopInputFormat #116
Date Sun, 08 Apr 2018 12:04:57 GMT
See <https://builds.apache.org/job/beam_PerformanceTests_HadoopInputFormat/116/display/redirect?page=changes>

Changes:

[relax] Rename RowType -> Schema and move to new schemas package.

[relax] Add classes for SchemaRegistry and SchemaProvider.

[relax] Add SchemaCoder, some more detail in SchemaProvider/Registery, and

[relax] Move Schema classes into SDK package.

[relax] Integrate Schemas into PCollection Coder inference.

[relax] Add setSchema convenience method.

[relax] Fix compilation errors, tests, and CheckStyle.

[relax] Add package-info.java for new schemas package.

[relax] Refactor Schema: Schema now is a fully-specified based on primitive

[relax] Make sure Schema class is serializable.

[relax] Add BigEndingShortCoder to encode int16s.

[relax] Cleanup types in Beam Schema. Add a metadata field to TypeDescriptor, so

[relax] Make all SQL fields nullable for now.

[relax] Add better type validation and conversion. Introduce FloatCoder.

[relax] Use Beam's built-in BooleanCoder.

[relax] Add coder mapping for DECIMAL.

[relax] Fix BigEndianShortCoder

[relax] Fix Array inference and capture.

[relax] Don't use byte[] as the key of a map - Java arrays use object equality.

[relax] Add equals and hashCode methods.

[relax] Remove unused code.

[relax] Fix handling of null values - Row was dropping them.

[relax] Allow field description to be null.

[relax] Fix array verification in Row.

[relax] Move BeamSQL over to use joda time.

[relax] Fix full outer join test.

[relax] Add more unit tests for Row, Schema, and RowCoder. Fix a few issues

[relax] Revert components that we are not ready to merge yet. These will

[relax] Sync ParDoTranslation to master.

[relax] Add license header.

[relax] Fix row size estimation.

[relax] Address code-review comments.

[relax] Add RowSqlTypes.

[relax] Make variables final and fix unused imports.

[relax] Fix RowCoderTest.

[relax] Delete bogus test.

[relax] Fix timezone test.

------------------------------------------
[...truncated 1.32 KB...]
Resetting working tree
 > git reset --hard # timeout=10
 > git clean -fdx # timeout=10
[EnvInject] - Executing scripts and injecting environment variables after the SCM step.
[EnvInject] - Injecting as environment variables the properties content 
SPARK_LOCAL_IP=127.0.0.1

[EnvInject] - Variables injected successfully.
[beam_PerformanceTests_HadoopInputFormat] $ /bin/bash -xe /tmp/jenkins4883664200266235606.sh
+ gcloud container clusters get-credentials io-datastores --zone=us-central1-a --verbosity=debug
DEBUG: Running gcloud.container.clusters.get-credentials with Namespace(__calliope_internal_deepest_parser=ArgumentParser(prog='gcloud.container.clusters.get-credentials',
usage=None, description='Updates a kubeconfig file with appropriate credentials to point\nkubectl
at a Container Engine Cluster. By default, credentials\nare written to HOME/.kube/config.
You can provide an alternate\npath by setting the KUBECONFIG environment variable.\n\nSee
[](https://cloud.google.com/container-engine/docs/kubectl) for\nkubectl documentation.', version=None,
formatter_class=<class 'argparse.HelpFormatter'>, conflict_handler='error', add_help=False),
account=None, api_version=None, authority_selector=None, authorization_token_file=None, calliope_command=<googlecloudsdk.calliope.backend.Command
object at 0x7fa0d068b050>, command_path=['gcloud', 'container', 'clusters', 'get-credentials'],
configuration=None, credential_file_override=None, document=None, flatten=None, format=None,
h=None, help=None, http_timeout=None, log_http=None, name='io-datastores', project=None, quiet=None,
trace_email=None, trace_log=None, trace_token=None, user_output_enabled=None, verbosity='debug',
version=None, zone='us-central1-a').
Fetching cluster endpoint and auth data.
DEBUG: Saved kubeconfig to /home/jenkins/.kube/config
kubeconfig entry generated for io-datastores.
INFO: Display format "default".
DEBUG: SDK update checks are disabled.
[beam_PerformanceTests_HadoopInputFormat] $ /bin/bash -xe /tmp/jenkins687257099703041553.sh
+ cp /home/jenkins/.kube/config <https://builds.apache.org/job/beam_PerformanceTests_HadoopInputFormat/ws/config-hadoopinputformatioit-1523181859821>
[beam_PerformanceTests_HadoopInputFormat] $ /bin/bash -xe /tmp/jenkins3657470212560833933.sh
+ kubectl --kubeconfig=<https://builds.apache.org/job/beam_PerformanceTests_HadoopInputFormat/ws/config-hadoopinputformatioit-1523181859821>
create namespace hadoopinputformatioit-1523181859821
namespace "hadoopinputformatioit-1523181859821" created
[beam_PerformanceTests_HadoopInputFormat] $ /bin/bash -xe /tmp/jenkins5308353381194342225.sh
++ kubectl config current-context
+ kubectl --kubeconfig=<https://builds.apache.org/job/beam_PerformanceTests_HadoopInputFormat/ws/config-hadoopinputformatioit-1523181859821>
config set-context gke_apache-beam-testing_us-central1-a_io-datastores --namespace=hadoopinputformatioit-1523181859821
Context "gke_apache-beam-testing_us-central1-a_io-datastores" modified.
[beam_PerformanceTests_HadoopInputFormat] $ /bin/bash -xe /tmp/jenkins7262718944588188841.sh
+ rm -rf PerfKitBenchmarker
[beam_PerformanceTests_HadoopInputFormat] $ /bin/bash -xe /tmp/jenkins3665237708861007302.sh
+ rm -rf .env
[beam_PerformanceTests_HadoopInputFormat] $ /bin/bash -xe /tmp/jenkins8755086737959823138.sh
+ virtualenv .env --system-site-packages
New python executable in .env/bin/python
Installing setuptools, pip...done.
[beam_PerformanceTests_HadoopInputFormat] $ /bin/bash -xe /tmp/jenkins6865379021389400830.sh
+ .env/bin/pip install --upgrade setuptools pip
Downloading/unpacking setuptools from https://pypi.python.org/packages/20/d7/04a0b689d3035143e2ff288f4b9ee4bf6ed80585cc121c90bfd85a1a8c2e/setuptools-39.0.1-py2.py3-none-any.whl#md5=ca299c7acd13a72e1171a3697f2b99bc
Downloading/unpacking pip from https://pypi.python.org/packages/ac/95/a05b56bb975efa78d3557efa36acaf9cf5d2fd0ee0062060493687432e03/pip-9.0.3-py2.py3-none-any.whl#md5=d512ceb964f38ba31addb8142bc657cb
Installing collected packages: setuptools, pip
  Found existing installation: setuptools 2.2
    Uninstalling setuptools:
      Successfully uninstalled setuptools
  Found existing installation: pip 1.5.4
    Uninstalling pip:
      Successfully uninstalled pip
Successfully installed setuptools pip
Cleaning up...
[beam_PerformanceTests_HadoopInputFormat] $ /bin/bash -xe /tmp/jenkins5001462170747338523.sh
+ git clone https://github.com/GoogleCloudPlatform/PerfKitBenchmarker.git
Cloning into 'PerfKitBenchmarker'...
[beam_PerformanceTests_HadoopInputFormat] $ /bin/bash -xe /tmp/jenkins52016805987463045.sh
+ .env/bin/pip install -r PerfKitBenchmarker/requirements.txt
Requirement already satisfied: absl-py in /home/jenkins/.local/lib/python2.7/site-packages
(from -r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied: jinja2>=2.7 in /usr/local/lib/python2.7/dist-packages (from
-r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied: setuptools in ./.env/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt
(line 16))
Requirement already satisfied: colorlog[windows]==2.6.0 in /home/jenkins/.local/lib/python2.7/site-packages
(from -r PerfKitBenchmarker/requirements.txt (line 17))
Requirement already satisfied: blinker>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages
(from -r PerfKitBenchmarker/requirements.txt (line 18))
Requirement already satisfied: futures>=3.0.3 in /home/jenkins/.local/lib/python2.7/site-packages
(from -r PerfKitBenchmarker/requirements.txt (line 19))
Requirement already satisfied: PyYAML==3.12 in /home/jenkins/.local/lib/python2.7/site-packages
(from -r PerfKitBenchmarker/requirements.txt (line 20))
Requirement already satisfied: pint>=0.7 in /home/jenkins/.local/lib/python2.7/site-packages
(from -r PerfKitBenchmarker/requirements.txt (line 21))
Collecting numpy==1.13.3 (from -r PerfKitBenchmarker/requirements.txt (line 22))
<https://builds.apache.org/job/beam_PerformanceTests_HadoopInputFormat/ws/.env/local/lib/python2.7/site-packages/pip/_vendor/urllib3/util/ssl_.py>:339:
SNIMissingWarning: An HTTPS request has been made, but the SNI (Subject Name Indication) extension
to TLS is not available on this platform. This may cause the server to present an incorrect
TLS certificate, which can cause validation failures. You can upgrade to a newer version of
Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings
  SNIMissingWarning
<https://builds.apache.org/job/beam_PerformanceTests_HadoopInputFormat/ws/.env/local/lib/python2.7/site-packages/pip/_vendor/urllib3/util/ssl_.py>:137:
InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3
from configuring SSL appropriately and may cause certain SSL connections to fail. You can
upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings
  InsecurePlatformWarning
  Using cached numpy-1.13.3-cp27-cp27mu-manylinux1_x86_64.whl
Requirement already satisfied: functools32 in /home/jenkins/.local/lib/python2.7/site-packages
(from -r PerfKitBenchmarker/requirements.txt (line 23))
Requirement already satisfied: contextlib2>=0.5.1 in /home/jenkins/.local/lib/python2.7/site-packages
(from -r PerfKitBenchmarker/requirements.txt (line 24))
Requirement already satisfied: pywinrm in /home/jenkins/.local/lib/python2.7/site-packages
(from -r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: six in /home/jenkins/.local/lib/python2.7/site-packages (from
absl-py->-r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied: MarkupSafe>=0.23 in /usr/local/lib/python2.7/dist-packages
(from jinja2>=2.7->-r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied: colorama; extra == "windows" in /usr/lib/python2.7/dist-packages
(from colorlog[windows]==2.6.0->-r PerfKitBenchmarker/requirements.txt (line 17))
Requirement already satisfied: xmltodict in /home/jenkins/.local/lib/python2.7/site-packages
(from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: requests-ntlm>=0.3.0 in /home/jenkins/.local/lib/python2.7/site-packages
(from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: requests>=2.9.1 in /home/jenkins/.local/lib/python2.7/site-packages
(from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: ntlm-auth>=1.0.2 in /home/jenkins/.local/lib/python2.7/site-packages
(from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line
25))
Requirement already satisfied: cryptography>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages
(from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line
25))
Requirement already satisfied: idna<2.6,>=2.5 in /home/jenkins/.local/lib/python2.7/site-packages
(from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: urllib3<1.22,>=1.21.1 in /home/jenkins/.local/lib/python2.7/site-packages
(from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /home/jenkins/.local/lib/python2.7/site-packages
(from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: certifi>=2017.4.17 in /home/jenkins/.local/lib/python2.7/site-packages
(from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: cffi>=1.7; platform_python_implementation != "PyPy" in /home/jenkins/.local/lib/python2.7/site-packages
(from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt
(line 25))
Requirement already satisfied: enum34; python_version < "3" in /home/jenkins/.local/lib/python2.7/site-packages
(from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt
(line 25))
Requirement already satisfied: asn1crypto>=0.21.0 in /home/jenkins/.local/lib/python2.7/site-packages
(from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt
(line 25))
Requirement already satisfied: ipaddress; python_version < "3" in /usr/local/lib/python2.7/dist-packages
(from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt
(line 25))
Requirement already satisfied: pycparser in /home/jenkins/.local/lib/python2.7/site-packages
(from cffi>=1.7; platform_python_implementation != "PyPy"->cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r
PerfKitBenchmarker/requirements.txt (line 25))
Installing collected packages: numpy
  Found existing installation: numpy 1.12.1
    Not uninstalling numpy at /home/jenkins/.local/lib/python2.7/site-packages, outside environment
<https://builds.apache.org/job/beam_PerformanceTests_HadoopInputFormat/ws/.env>
Successfully installed numpy-1.13.3
<https://builds.apache.org/job/beam_PerformanceTests_HadoopInputFormat/ws/.env/local/lib/python2.7/site-packages/pip/_vendor/urllib3/util/ssl_.py>:137:
InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3
from configuring SSL appropriately and may cause certain SSL connections to fail. You can
upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings
  InsecurePlatformWarning
[beam_PerformanceTests_HadoopInputFormat] $ /bin/bash -xe /tmp/jenkins2819685708599356775.sh
+ .env/bin/pip install -e 'src/sdks/python/[gcp,test]'
Obtaining file://<https://builds.apache.org/job/beam_PerformanceTests_HadoopInputFormat/ws/src/sdks/python>
Requirement already satisfied: avro<2.0.0,>=1.8.1 in /home/jenkins/.local/lib/python2.7/site-packages
(from apache-beam==2.5.0.dev0)
Requirement already satisfied: crcmod<2.0,>=1.7 in /usr/lib/python2.7/dist-packages
(from apache-beam==2.5.0.dev0)
Requirement already satisfied: dill==0.2.6 in /home/jenkins/.local/lib/python2.7/site-packages
(from apache-beam==2.5.0.dev0)
Collecting grpcio<2,>=1.8 (from apache-beam==2.5.0.dev0)
<https://builds.apache.org/job/beam_PerformanceTests_HadoopInputFormat/ws/.env/local/lib/python2.7/site-packages/pip/_vendor/urllib3/util/ssl_.py>:339:
SNIMissingWarning: An HTTPS request has been made, but the SNI (Subject Name Indication) extension
to TLS is not available on this platform. This may cause the server to present an incorrect
TLS certificate, which can cause validation failures. You can upgrade to a newer version of
Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings
  SNIMissingWarning
<https://builds.apache.org/job/beam_PerformanceTests_HadoopInputFormat/ws/.env/local/lib/python2.7/site-packages/pip/_vendor/urllib3/util/ssl_.py>:137:
InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3
from configuring SSL appropriately and may cause certain SSL connections to fail. You can
upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings
  InsecurePlatformWarning
  Using cached grpcio-1.10.1-cp27-cp27mu-manylinux1_x86_64.whl
Collecting hdfs<3.0.0,>=2.1.0 (from apache-beam==2.5.0.dev0)
Requirement already satisfied: httplib2<0.10,>=0.8 in /home/jenkins/.local/lib/python2.7/site-packages
(from apache-beam==2.5.0.dev0)
Requirement already satisfied: mock<3.0.0,>=1.0.1 in /home/jenkins/.local/lib/python2.7/site-packages
(from apache-beam==2.5.0.dev0)
Requirement already satisfied: oauth2client<5,>=2.0.1 in /home/jenkins/.local/lib/python2.7/site-packages
(from apache-beam==2.5.0.dev0)
Requirement already satisfied: protobuf<4,>=3.5.0.post1 in /home/jenkins/.local/lib/python2.7/site-packages
(from apache-beam==2.5.0.dev0)
Collecting pytz>=2018.3 (from apache-beam==2.5.0.dev0)
  Using cached pytz-2018.3-py2.py3-none-any.whl
Requirement already satisfied: pyyaml<4.0.0,>=3.12 in /home/jenkins/.local/lib/python2.7/site-packages
(from apache-beam==2.5.0.dev0)
Requirement already satisfied: pyvcf<0.7.0,>=0.6.8 in /home/jenkins/.local/lib/python2.7/site-packages
(from apache-beam==2.5.0.dev0)
Requirement already satisfied: six<1.12,>=1.9 in /home/jenkins/.local/lib/python2.7/site-packages
(from apache-beam==2.5.0.dev0)
Requirement already satisfied: typing<3.7.0,>=3.6.0 in /home/jenkins/.local/lib/python2.7/site-packages
(from apache-beam==2.5.0.dev0)
Requirement already satisfied: futures<4.0.0,>=3.1.1 in /home/jenkins/.local/lib/python2.7/site-packages
(from apache-beam==2.5.0.dev0)
Requirement already satisfied: google-apitools<=0.5.20,>=0.5.18 in /home/jenkins/.local/lib/python2.7/site-packages
(from apache-beam==2.5.0.dev0)
Requirement already satisfied: proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0 in /home/jenkins/.local/lib/python2.7/site-packages
(from apache-beam==2.5.0.dev0)
Requirement already satisfied: googledatastore==7.0.1 in /home/jenkins/.local/lib/python2.7/site-packages
(from apache-beam==2.5.0.dev0)
Requirement already satisfied: google-cloud-pubsub==0.26.0 in /home/jenkins/.local/lib/python2.7/site-packages
(from apache-beam==2.5.0.dev0)
Requirement already satisfied: proto-google-cloud-pubsub-v1==0.15.4 in /home/jenkins/.local/lib/python2.7/site-packages
(from apache-beam==2.5.0.dev0)
Requirement already satisfied: google-cloud-bigquery==0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages
(from apache-beam==2.5.0.dev0)
Collecting nose>=1.3.7 (from apache-beam==2.5.0.dev0)
  Using cached nose-1.3.7-py2-none-any.whl
Requirement already satisfied: pyhamcrest<2.0,>=1.9 in /home/jenkins/.local/lib/python2.7/site-packages
(from apache-beam==2.5.0.dev0)
Requirement already satisfied: enum34>=1.0.4 in /home/jenkins/.local/lib/python2.7/site-packages
(from grpcio<2,>=1.8->apache-beam==2.5.0.dev0)
Requirement already satisfied: requests>=2.7.0 in /home/jenkins/.local/lib/python2.7/site-packages
(from hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0)
Requirement already satisfied: docopt in /usr/local/lib/python2.7/dist-packages (from hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0)
Requirement already satisfied: funcsigs>=1; python_version < "3.3" in /home/jenkins/.local/lib/python2.7/site-packages
(from mock<3.0.0,>=1.0.1->apache-beam==2.5.0.dev0)
Requirement already satisfied: pbr>=0.11 in /home/jenkins/.local/lib/python2.7/site-packages
(from mock<3.0.0,>=1.0.1->apache-beam==2.5.0.dev0)
Requirement already satisfied: pyasn1>=0.1.7 in /home/jenkins/.local/lib/python2.7/site-packages
(from oauth2client<5,>=2.0.1->apache-beam==2.5.0.dev0)
Requirement already satisfied: pyasn1-modules>=0.0.5 in /home/jenkins/.local/lib/python2.7/site-packages
(from oauth2client<5,>=2.0.1->apache-beam==2.5.0.dev0)
Requirement already satisfied: rsa>=3.1.4 in /home/jenkins/.local/lib/python2.7/site-packages
(from oauth2client<5,>=2.0.1->apache-beam==2.5.0.dev0)
Requirement already satisfied: setuptools in ./.env/lib/python2.7/site-packages (from protobuf<4,>=3.5.0.post1->apache-beam==2.5.0.dev0)
Requirement already satisfied: fasteners>=0.14 in /home/jenkins/.local/lib/python2.7/site-packages
(from google-apitools<=0.5.20,>=0.5.18->apache-beam==2.5.0.dev0)
Requirement already satisfied: googleapis-common-protos<2.0dev,>=1.5.2 in /home/jenkins/.local/lib/python2.7/site-packages
(from proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0->apache-beam==2.5.0.dev0)
Requirement already satisfied: google-cloud-core<0.26dev,>=0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages
(from google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Requirement already satisfied: gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0 in /home/jenkins/.local/lib/python2.7/site-packages
(from google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Requirement already satisfied: idna<2.6,>=2.5 in /home/jenkins/.local/lib/python2.7/site-packages
(from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0)
Requirement already satisfied: urllib3<1.22,>=1.21.1 in /home/jenkins/.local/lib/python2.7/site-packages
(from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /home/jenkins/.local/lib/python2.7/site-packages
(from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0)
Requirement already satisfied: certifi>=2017.4.17 in /home/jenkins/.local/lib/python2.7/site-packages
(from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0)
Requirement already satisfied: monotonic>=0.1 in /home/jenkins/.local/lib/python2.7/site-packages
(from fasteners>=0.14->google-apitools<=0.5.20,>=0.5.18->apache-beam==2.5.0.dev0)
Requirement already satisfied: google-auth<2.0.0dev,>=0.4.0 in /home/jenkins/.local/lib/python2.7/site-packages
(from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Requirement already satisfied: google-auth-httplib2 in /home/jenkins/.local/lib/python2.7/site-packages
(from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Requirement already satisfied: google-gax<0.16dev,>=0.15.7 in /home/jenkins/.local/lib/python2.7/site-packages
(from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Requirement already satisfied: grpc-google-iam-v1<0.12dev,>=0.11.1 in /home/jenkins/.local/lib/python2.7/site-packages
(from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Requirement already satisfied: cachetools>=2.0.0 in /home/jenkins/.local/lib/python2.7/site-packages
(from google-auth<2.0.0dev,>=0.4.0->google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Requirement already satisfied: future<0.17dev,>=0.16.0 in /home/jenkins/.local/lib/python2.7/site-packages
(from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Requirement already satisfied: ply==3.8 in /home/jenkins/.local/lib/python2.7/site-packages
(from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Installing collected packages: grpcio, hdfs, pytz, nose, apache-beam
  Found existing installation: grpcio 1.4.0
    Not uninstalling grpcio at /home/jenkins/.local/lib/python2.7/site-packages, outside environment
<https://builds.apache.org/job/beam_PerformanceTests_HadoopInputFormat/ws/.env>
  Found existing installation: apache-beam 2.5.0.dev0
    Not uninstalling apache-beam at /home/jenkins/jenkins-slave/workspace/beam_PerformanceTests_AvroIOIT/src/sdks/python,
outside environment <https://builds.apache.org/job/beam_PerformanceTests_HadoopInputFormat/ws/.env>
  Running setup.py develop for apache-beam
Successfully installed apache-beam grpcio-1.10.1 hdfs-2.1.0 nose-1.3.7 pytz-2018.3
[beam_PerformanceTests_HadoopInputFormat] $ /bin/bash -xe /tmp/jenkins8377985161464402032.sh
+ .env/bin/python PerfKitBenchmarker/pkb.py --project=apache-beam-testing --dpb_log_level=INFO
--maven_binary=/home/jenkins/tools/maven/latest/bin/mvn --bigquery_table=beam_performance.hadoopinputformatioit_pkb_results
--temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_HadoopInputFormat/ws/>
--official=true --kubeconfig=<https://builds.apache.org/job/beam_PerformanceTests_HadoopInputFormat/ws/config-hadoopinputformatioit-1523181859821>
--beam_it_timeout=1200 --benchmarks=beam_integration_benchmark --beam_it_profile=io-it --beam_prebuilt=true
--beam_sdk=java --beam_it_module=sdks/java/io/hadoop-input-format --beam_it_class=org.apache.beam.sdk.io.hadoop.inputformat.HadoopInputFormatIOIT
'--beam_it_options=[--tempRoot=gs://temp-storage-for-perf-tests,--project=apache-beam-testing,--postgresPort=5432,--numberOfRecords=600000]'
--beam_kubernetes_scripts=<https://builds.apache.org/job/beam_PerformanceTests_HadoopInputFormat/ws/src/.test-infra/kubernetes/postgres/postgres-service-for-local-dev.yml>
--beam_options_config_file=<https://builds.apache.org/job/beam_PerformanceTests_HadoopInputFormat/ws/src/.test-infra/kubernetes/postgres/pkb-config-local.yml>
2018-04-08 12:01:44,021 2a67d52f MainThread INFO     Verbose logging to: <https://builds.apache.org/job/beam_PerformanceTests_HadoopInputFormat/ws/runs/2a67d52f/pkb.log>
2018-04-08 12:01:44,021 2a67d52f MainThread INFO     PerfKitBenchmarker version: v1.12.0-468-gd19e6df
2018-04-08 12:01:44,022 2a67d52f MainThread INFO     Flag values:
--beam_it_class=org.apache.beam.sdk.io.hadoop.inputformat.HadoopInputFormatIOIT
--beam_it_timeout=1200
--beam_sdk=java
--temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_HadoopInputFormat/ws/>
--maven_binary=/home/jenkins/tools/maven/latest/bin/mvn
--beam_it_options=[--tempRoot=gs://temp-storage-for-perf-tests,--project=apache-beam-testing,--postgresPort=5432,--numberOfRecords=600000]
--beam_prebuilt
--kubeconfig=<https://builds.apache.org/job/beam_PerformanceTests_HadoopInputFormat/ws/config-hadoopinputformatioit-1523181859821>
--project=apache-beam-testing
--beam_kubernetes_scripts=<https://builds.apache.org/job/beam_PerformanceTests_HadoopInputFormat/ws/src/.test-infra/kubernetes/postgres/postgres-service-for-local-dev.yml>
--bigquery_table=beam_performance.hadoopinputformatioit_pkb_results
--official
--beam_it_module=sdks/java/io/hadoop-input-format
--dpb_log_level=INFO
--beam_options_config_file=<https://builds.apache.org/job/beam_PerformanceTests_HadoopInputFormat/ws/src/.test-infra/kubernetes/postgres/pkb-config-local.yml>
--beam_it_profile=io-it
--benchmarks=beam_integration_benchmark
2018-04-08 12:01:44,349 2a67d52f MainThread WARNING  The key "flags" was not in the default
config, but was in user overrides. This may indicate a typo.
2018-04-08 12:01:44,349 2a67d52f MainThread INFO     Initializing the edw service decoder
2018-04-08 12:01:44,523 2a67d52f MainThread beam_integration_benchmark(1/1) INFO     Provisioning
resources for benchmark beam_integration_benchmark
2018-04-08 12:01:44,526 2a67d52f MainThread beam_integration_benchmark(1/1) INFO     Preparing
benchmark beam_integration_benchmark
2018-04-08 12:01:44,526 2a67d52f MainThread beam_integration_benchmark(1/1) INFO     Running:
git clone https://github.com/apache/beam.git
2018-04-08 12:01:52,601 2a67d52f MainThread beam_integration_benchmark(1/1) INFO     Running:
kubectl --kubeconfig=<https://builds.apache.org/job/beam_PerformanceTests_HadoopInputFormat/ws/config-hadoopinputformatioit-1523181859821>
create -f <https://builds.apache.org/job/beam_PerformanceTests_HadoopInputFormat/ws/src/.test-infra/kubernetes/postgres/postgres-service-for-local-dev.yml>
2018-04-08 12:01:52,980 2a67d52f MainThread beam_integration_benchmark(1/1) INFO     Running
benchmark beam_integration_benchmark
2018-04-08 12:01:53,009 2a67d52f MainThread beam_integration_benchmark(1/1) INFO     Running:
kubectl --kubeconfig=<https://builds.apache.org/job/beam_PerformanceTests_HadoopInputFormat/ws/config-hadoopinputformatioit-1523181859821>
get svc postgres-for-dev -ojsonpath={.status.loadBalancer.ingress[0].ip}
2018-04-08 12:02:03,198 2a67d52f MainThread beam_integration_benchmark(1/1) INFO     Running:
kubectl --kubeconfig=<https://builds.apache.org/job/beam_PerformanceTests_HadoopInputFormat/ws/config-hadoopinputformatioit-1523181859821>
get svc postgres-for-dev -ojsonpath={.status.loadBalancer.ingress[0].ip}
2018-04-08 12:02:13,341 2a67d52f MainThread beam_integration_benchmark(1/1) INFO     Running:
kubectl --kubeconfig=<https://builds.apache.org/job/beam_PerformanceTests_HadoopInputFormat/ws/config-hadoopinputformatioit-1523181859821>
get svc postgres-for-dev -ojsonpath={.status.loadBalancer.ingress[0].ip}
2018-04-08 12:02:23,553 2a67d52f MainThread beam_integration_benchmark(1/1) INFO     Running:
kubectl --kubeconfig=<https://builds.apache.org/job/beam_PerformanceTests_HadoopInputFormat/ws/config-hadoopinputformatioit-1523181859821>
get svc postgres-for-dev -ojsonpath={.status.loadBalancer.ingress[0].ip}
2018-04-08 12:02:33,721 2a67d52f MainThread beam_integration_benchmark(1/1) INFO     Running:
kubectl --kubeconfig=<https://builds.apache.org/job/beam_PerformanceTests_HadoopInputFormat/ws/config-hadoopinputformatioit-1523181859821>
get svc postgres-for-dev -ojsonpath={.status.loadBalancer.ingress[0].ip}
2018-04-08 12:02:43,883 2a67d52f MainThread beam_integration_benchmark(1/1) INFO     Running:
kubectl --kubeconfig=<https://builds.apache.org/job/beam_PerformanceTests_HadoopInputFormat/ws/config-hadoopinputformatioit-1523181859821>
get svc postgres-for-dev -ojsonpath={.status.loadBalancer.ingress[0].ip}
2018-04-08 12:02:54,022 2a67d52f MainThread beam_integration_benchmark(1/1) INFO     Running:
kubectl --kubeconfig=<https://builds.apache.org/job/beam_PerformanceTests_HadoopInputFormat/ws/config-hadoopinputformatioit-1523181859821>
get svc postgres-for-dev -ojsonpath={.status.loadBalancer.ingress[0].ip}
2018-04-08 12:03:04,166 2a67d52f MainThread beam_integration_benchmark(1/1) INFO     Running:
kubectl --kubeconfig=<https://builds.apache.org/job/beam_PerformanceTests_HadoopInputFormat/ws/config-hadoopinputformatioit-1523181859821>
get svc postgres-for-dev -ojsonpath={.status.loadBalancer.ingress[0].ip}
2018-04-08 12:03:14,308 2a67d52f MainThread beam_integration_benchmark(1/1) INFO     Running:
kubectl --kubeconfig=<https://builds.apache.org/job/beam_PerformanceTests_HadoopInputFormat/ws/config-hadoopinputformatioit-1523181859821>
get svc postgres-for-dev -ojsonpath={.status.loadBalancer.ingress[0].ip}
2018-04-08 12:03:24,463 2a67d52f MainThread beam_integration_benchmark(1/1) INFO     Running:
kubectl --kubeconfig=<https://builds.apache.org/job/beam_PerformanceTests_HadoopInputFormat/ws/config-hadoopinputformatioit-1523181859821>
get svc postgres-for-dev -ojsonpath={.status.loadBalancer.ingress[0].ip}
2018-04-08 12:03:34,665 2a67d52f MainThread beam_integration_benchmark(1/1) INFO     Running:
kubectl --kubeconfig=<https://builds.apache.org/job/beam_PerformanceTests_HadoopInputFormat/ws/config-hadoopinputformatioit-1523181859821>
get svc postgres-for-dev -ojsonpath={.status.loadBalancer.ingress[0].ip}
2018-04-08 12:03:44,824 2a67d52f MainThread beam_integration_benchmark(1/1) INFO     Running:
kubectl --kubeconfig=<https://builds.apache.org/job/beam_PerformanceTests_HadoopInputFormat/ws/config-hadoopinputformatioit-1523181859821>
get svc postgres-for-dev -ojsonpath={.status.loadBalancer.ingress[0].ip}
2018-04-08 12:03:54,996 2a67d52f MainThread beam_integration_benchmark(1/1) INFO     Running:
kubectl --kubeconfig=<https://builds.apache.org/job/beam_PerformanceTests_HadoopInputFormat/ws/config-hadoopinputformatioit-1523181859821>
get svc postgres-for-dev -ojsonpath={.status.loadBalancer.ingress[0].ip}
2018-04-08 12:04:05,150 2a67d52f MainThread beam_integration_benchmark(1/1) INFO     Running:
kubectl --kubeconfig=<https://builds.apache.org/job/beam_PerformanceTests_HadoopInputFormat/ws/config-hadoopinputformatioit-1523181859821>
get svc postgres-for-dev -ojsonpath={.status.loadBalancer.ingress[0].ip}
2018-04-08 12:04:15,306 2a67d52f MainThread beam_integration_benchmark(1/1) INFO     Running:
kubectl --kubeconfig=<https://builds.apache.org/job/beam_PerformanceTests_HadoopInputFormat/ws/config-hadoopinputformatioit-1523181859821>
get svc postgres-for-dev -ojsonpath={.status.loadBalancer.ingress[0].ip}
2018-04-08 12:04:25,463 2a67d52f MainThread beam_integration_benchmark(1/1) INFO     Running:
kubectl --kubeconfig=<https://builds.apache.org/job/beam_PerformanceTests_HadoopInputFormat/ws/config-hadoopinputformatioit-1523181859821>
get svc postgres-for-dev -ojsonpath={.status.loadBalancer.ingress[0].ip}
2018-04-08 12:04:35,600 2a67d52f MainThread beam_integration_benchmark(1/1) INFO     Running:
kubectl --kubeconfig=<https://builds.apache.org/job/beam_PerformanceTests_HadoopInputFormat/ws/config-hadoopinputformatioit-1523181859821>
get svc postgres-for-dev -ojsonpath={.status.loadBalancer.ingress[0].ip}
2018-04-08 12:04:45,746 2a67d52f MainThread beam_integration_benchmark(1/1) INFO     Running:
kubectl --kubeconfig=<https://builds.apache.org/job/beam_PerformanceTests_HadoopInputFormat/ws/config-hadoopinputformatioit-1523181859821>
get svc postgres-for-dev -ojsonpath={.status.loadBalancer.ingress[0].ip}
2018-04-08 12:04:55,883 2a67d52f MainThread beam_integration_benchmark(1/1) INFO     Running:
kubectl --kubeconfig=<https://builds.apache.org/job/beam_PerformanceTests_HadoopInputFormat/ws/config-hadoopinputformatioit-1523181859821>
get svc postgres-for-dev -ojsonpath={.status.loadBalancer.ingress[0].ip}
2018-04-08 12:04:56,029 2a67d52f MainThread beam_integration_benchmark(1/1) ERROR    Error
during benchmark beam_integration_benchmark
Traceback (most recent call last):
  File "<https://builds.apache.org/job/beam_PerformanceTests_HadoopInputFormat/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",>
line 624, in RunBenchmark
    DoRunPhase(spec, collector, detailed_timer)
  File "<https://builds.apache.org/job/beam_PerformanceTests_HadoopInputFormat/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",>
line 527, in DoRunPhase
    samples = spec.BenchmarkRun(spec)
  File "<https://builds.apache.org/job/beam_PerformanceTests_HadoopInputFormat/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",>
line 148, in Run
    dynamic_pipeline_options)
  File "<https://builds.apache.org/job/beam_PerformanceTests_HadoopInputFormat/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_pipeline_options.py",>
line 100, in GenerateAllPipelineOptions
    EvaluateDynamicPipelineOptions(dynamic_pipeline_options))
  File "<https://builds.apache.org/job/beam_PerformanceTests_HadoopInputFormat/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_pipeline_options.py",>
line 67, in EvaluateDynamicPipelineOptions
    argValue = RetrieveLoadBalancerIp(optionDescriptor)
  File "<https://builds.apache.org/job/beam_PerformanceTests_HadoopInputFormat/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_pipeline_options.py",>
line 153, in RetrieveLoadBalancerIp
    raise "Could not retrieve LoadBalancer IP address"
TypeError: exceptions must be old-style classes or derived from BaseException, not str
2018-04-08 12:04:56,030 2a67d52f MainThread beam_integration_benchmark(1/1) INFO     Cleaning
up benchmark beam_integration_benchmark
2018-04-08 12:04:56,030 2a67d52f MainThread beam_integration_benchmark(1/1) INFO     Running:
kubectl --kubeconfig=<https://builds.apache.org/job/beam_PerformanceTests_HadoopInputFormat/ws/config-hadoopinputformatioit-1523181859821>
delete -f <https://builds.apache.org/job/beam_PerformanceTests_HadoopInputFormat/ws/src/.test-infra/kubernetes/postgres/postgres-service-for-local-dev.yml>
2018-04-08 12:04:56,529 2a67d52f MainThread beam_integration_benchmark(1/1) ERROR    Exception
running benchmark
Traceback (most recent call last):
  File "<https://builds.apache.org/job/beam_PerformanceTests_HadoopInputFormat/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",>
line 758, in RunBenchmarkTask
    RunBenchmark(spec, collector)
  File "<https://builds.apache.org/job/beam_PerformanceTests_HadoopInputFormat/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",>
line 624, in RunBenchmark
    DoRunPhase(spec, collector, detailed_timer)
  File "<https://builds.apache.org/job/beam_PerformanceTests_HadoopInputFormat/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",>
line 527, in DoRunPhase
    samples = spec.BenchmarkRun(spec)
  File "<https://builds.apache.org/job/beam_PerformanceTests_HadoopInputFormat/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",>
line 148, in Run
    dynamic_pipeline_options)
  File "<https://builds.apache.org/job/beam_PerformanceTests_HadoopInputFormat/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_pipeline_options.py",>
line 100, in GenerateAllPipelineOptions
    EvaluateDynamicPipelineOptions(dynamic_pipeline_options))
  File "<https://builds.apache.org/job/beam_PerformanceTests_HadoopInputFormat/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_pipeline_options.py",>
line 67, in EvaluateDynamicPipelineOptions
    argValue = RetrieveLoadBalancerIp(optionDescriptor)
  File "<https://builds.apache.org/job/beam_PerformanceTests_HadoopInputFormat/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_pipeline_options.py",>
line 153, in RetrieveLoadBalancerIp
    raise "Could not retrieve LoadBalancer IP address"
TypeError: exceptions must be old-style classes or derived from BaseException, not str
2018-04-08 12:04:56,529 2a67d52f MainThread beam_integration_benchmark(1/1) ERROR    Benchmark
1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2018-04-08 12:04:56,571 2a67d52f MainThread INFO     Benchmark run statuses:
---------------------------------------------------------------------------------
Name                        UID                          Status  Failed Substatus
---------------------------------------------------------------------------------
beam_integration_benchmark  beam_integration_benchmark0  FAILED                  
---------------------------------------------------------------------------------
Success rate: 0.00% (0/1)
2018-04-08 12:04:56,572 2a67d52f MainThread INFO     Complete logs can be found at: <https://builds.apache.org/job/beam_PerformanceTests_HadoopInputFormat/ws/runs/2a67d52f/pkb.log>
2018-04-08 12:04:56,572 2a67d52f MainThread INFO     Completion statuses can be found at:
<https://builds.apache.org/job/beam_PerformanceTests_HadoopInputFormat/ws/runs/2a67d52f/completion_statuses.json>
Build step 'Execute shell' marked build as failure


Mime
View raw message