beam-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Apache Jenkins Server <jenk...@builds.apache.org>
Subject Build failed in Jenkins: beam_PerformanceTests_ParquetIOIT_HDFS #179
Date Thu, 26 Jul 2018 12:22:33 GMT
See <https://builds.apache.org/job/beam_PerformanceTests_ParquetIOIT_HDFS/179/display/redirect?page=changes>

Changes:

[dariusz.aniszewski] separate virtualenvs for Beam and PerfKit

------------------------------------------
Started by timer
[EnvInject] - Loading node environment variables.
Building remotely on beam8 (beam) in workspace <https://builds.apache.org/job/beam_PerformanceTests_ParquetIOIT_HDFS/ws/>
 > git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
 > git config remote.origin.url https://github.com/apache/beam.git # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
 > git --version # timeout=10
 > git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/*
+refs/pull/${ghprbPullId}/*:refs/remotes/origin/pr/${ghprbPullId}/*
 > git rev-parse origin/master^{commit} # timeout=10
Checking out Revision aa2b585a5356ce3b607dd3d17a3c6ab2ce65353b (origin/master)
 > git config core.sparsecheckout # timeout=10
 > git checkout -f aa2b585a5356ce3b607dd3d17a3c6ab2ce65353b
Commit message: "Merge pull request #6056: [BEAM-4836] separate virtualenvs for Beam and PerfKit"
 > git rev-list --no-walk dd5de56908f31a80bc05bae9e6b7ae0251ae9b54 # timeout=10
Cleaning workspace
 > git rev-parse --verify HEAD # timeout=10
Resetting working tree
 > git reset --hard # timeout=10
 > git clean -fdx # timeout=10
[EnvInject] - Executing scripts and injecting environment variables after the SCM step.
[EnvInject] - Injecting as environment variables the properties content 
SPARK_LOCAL_IP=127.0.0.1

[EnvInject] - Variables injected successfully.
[beam_PerformanceTests_ParquetIOIT_HDFS] $ /bin/bash -xe /tmp/jenkins2827658572666763169.sh
+ gcloud container clusters get-credentials io-datastores --zone=us-central1-a --verbosity=debug
DEBUG: Running [gcloud.container.clusters.get-credentials] with arguments: [--verbosity: "debug",
--zone: "us-central1-a", NAME: "io-datastores"]
Fetching cluster endpoint and auth data.
DEBUG: Saved kubeconfig to /home/jenkins/.kube/config
kubeconfig entry generated for io-datastores.
INFO: Display format "default".
DEBUG: SDK update checks are disabled.
[beam_PerformanceTests_ParquetIOIT_HDFS] $ /bin/bash -xe /tmp/jenkins5423776978049332143.sh
+ cp /home/jenkins/.kube/config <https://builds.apache.org/job/beam_PerformanceTests_ParquetIOIT_HDFS/ws/config-beam-performancetests-parquetioit-hdfs-179>
[beam_PerformanceTests_ParquetIOIT_HDFS] $ /bin/bash -xe /tmp/jenkins5817849901944167964.sh
+ kubectl --kubeconfig=<https://builds.apache.org/job/beam_PerformanceTests_ParquetIOIT_HDFS/ws/config-beam-performancetests-parquetioit-hdfs-179>
create namespace beam-performancetests-parquetioit-hdfs-179
namespace "beam-performancetests-parquetioit-hdfs-179" created
[beam_PerformanceTests_ParquetIOIT_HDFS] $ /bin/bash -xe /tmp/jenkins501172349434955949.sh
++ kubectl config current-context
+ kubectl --kubeconfig=<https://builds.apache.org/job/beam_PerformanceTests_ParquetIOIT_HDFS/ws/config-beam-performancetests-parquetioit-hdfs-179>
config set-context gke_apache-beam-testing_us-central1-a_io-datastores --namespace=beam-performancetests-parquetioit-hdfs-179
Context "gke_apache-beam-testing_us-central1-a_io-datastores" modified.
[beam_PerformanceTests_ParquetIOIT_HDFS] $ /bin/bash -xe /tmp/jenkins3755597052676018388.sh
+ rm -rf PerfKitBenchmarker
[beam_PerformanceTests_ParquetIOIT_HDFS] $ /bin/bash -xe /tmp/jenkins2961324096996296429.sh
+ rm -rf .beam_env
[beam_PerformanceTests_ParquetIOIT_HDFS] $ /bin/bash -xe /tmp/jenkins6691183418516573822.sh
+ rm -rf .perfkit_env
[beam_PerformanceTests_ParquetIOIT_HDFS] $ /bin/bash -xe /tmp/jenkins7352790927994249455.sh
+ virtualenv .beam_env --system-site-packages
New python executable in <https://builds.apache.org/job/beam_PerformanceTests_ParquetIOIT_HDFS/ws/.beam_env/bin/python2>
Also creating executable in <https://builds.apache.org/job/beam_PerformanceTests_ParquetIOIT_HDFS/ws/.beam_env/bin/python>
Installing setuptools, pkg_resources, pip, wheel...done.
Running virtualenv with interpreter /usr/bin/python2
[beam_PerformanceTests_ParquetIOIT_HDFS] $ /bin/bash -xe /tmp/jenkins1693551862780902537.sh
+ virtualenv .perfkit_env --system-site-packages
New python executable in <https://builds.apache.org/job/beam_PerformanceTests_ParquetIOIT_HDFS/ws/.perfkit_env/bin/python2>
Also creating executable in <https://builds.apache.org/job/beam_PerformanceTests_ParquetIOIT_HDFS/ws/.perfkit_env/bin/python>
Installing setuptools, pkg_resources, pip, wheel...done.
Running virtualenv with interpreter /usr/bin/python2
[beam_PerformanceTests_ParquetIOIT_HDFS] $ /bin/bash -xe /tmp/jenkins908064090565028438.sh
+ .beam_env/bin/pip install --upgrade setuptools pip
Requirement already up-to-date: setuptools in ./.beam_env/lib/python2.7/site-packages (40.0.0)
Requirement already up-to-date: pip in ./.beam_env/lib/python2.7/site-packages (18.0)
[beam_PerformanceTests_ParquetIOIT_HDFS] $ /bin/bash -xe /tmp/jenkins4535586809290636558.sh
+ .perfkit_env/bin/pip install --upgrade setuptools pip
Requirement already up-to-date: setuptools in ./.perfkit_env/lib/python2.7/site-packages (40.0.0)
Requirement already up-to-date: pip in ./.perfkit_env/lib/python2.7/site-packages (18.0)
[beam_PerformanceTests_ParquetIOIT_HDFS] $ /bin/bash -xe /tmp/jenkins2122233103721141968.sh
+ git clone https://github.com/GoogleCloudPlatform/PerfKitBenchmarker.git
Cloning into 'PerfKitBenchmarker'...
[beam_PerformanceTests_ParquetIOIT_HDFS] $ /bin/bash -xe /tmp/jenkins8389259138725161276.sh
+ .beam_env/bin/pip install -e 'src/sdks/python/[gcp,test]'
Obtaining file://<https://builds.apache.org/job/beam_PerformanceTests_ParquetIOIT_HDFS/ws/src/sdks/python>
Requirement already satisfied: avro<2.0.0,>=1.8.1 in /home/jenkins/.local/lib/python2.7/site-packages
(from apache-beam==2.7.0.dev0) (1.8.2)
Requirement already satisfied: crcmod<2.0,>=1.7 in /usr/lib/python2.7/dist-packages
(from apache-beam==2.7.0.dev0) (1.7)
Requirement already satisfied: dill<=0.2.8.2,>=0.2.6 in /home/jenkins/.local/lib/python2.7/site-packages
(from apache-beam==2.7.0.dev0) (0.2.8.2)
Requirement already satisfied: grpcio<2,>=1.8 in /home/jenkins/.local/lib/python2.7/site-packages
(from apache-beam==2.7.0.dev0) (1.13.0)
Requirement already satisfied: hdfs<3.0.0,>=2.1.0 in /home/jenkins/.local/lib/python2.7/site-packages
(from apache-beam==2.7.0.dev0) (2.1.0)
Requirement already satisfied: httplib2<=0.11.3,>=0.8 in /home/jenkins/.local/lib/python2.7/site-packages
(from apache-beam==2.7.0.dev0) (0.11.3)
Requirement already satisfied: mock<3.0.0,>=1.0.1 in /home/jenkins/.local/lib/python2.7/site-packages
(from apache-beam==2.7.0.dev0) (2.0.0)
Requirement already satisfied: oauth2client<5,>=2.0.1 in /home/jenkins/.local/lib/python2.7/site-packages
(from apache-beam==2.7.0.dev0) (4.1.2)
Requirement already satisfied: protobuf<4,>=3.5.0.post1 in /home/jenkins/.local/lib/python2.7/site-packages
(from apache-beam==2.7.0.dev0) (3.6.0)
Requirement already satisfied: pydot<1.3,>=1.2.0 in /home/jenkins/.local/lib/python2.7/site-packages
(from apache-beam==2.7.0.dev0) (1.2.4)
Requirement already satisfied: pytz<=2018.4,>=2018.3 in /home/jenkins/.local/lib/python2.7/site-packages
(from apache-beam==2.7.0.dev0) (2018.4)
Requirement already satisfied: pyyaml<4.0.0,>=3.12 in /usr/local/lib/python2.7/dist-packages
(from apache-beam==2.7.0.dev0) (3.12)
Requirement already satisfied: pyvcf<0.7.0,>=0.6.8 in /home/jenkins/.local/lib/python2.7/site-packages
(from apache-beam==2.7.0.dev0) (0.6.8)
Requirement already satisfied: six<1.12,>=1.9 in /usr/local/lib/python2.7/dist-packages
(from apache-beam==2.7.0.dev0) (1.11.0)
Requirement already satisfied: typing<3.7.0,>=3.6.0 in /home/jenkins/.local/lib/python2.7/site-packages
(from apache-beam==2.7.0.dev0) (3.6.4)
Requirement already satisfied: futures<4.0.0,>=3.1.1 in /home/jenkins/.local/lib/python2.7/site-packages
(from apache-beam==2.7.0.dev0) (3.2.0)
Requirement already satisfied: future<1.0.0,>=0.16.0 in /home/jenkins/.local/lib/python2.7/site-packages
(from apache-beam==2.7.0.dev0) (0.16.0)
Requirement already satisfied: fastavro==0.19.7 in /home/jenkins/.local/lib/python2.7/site-packages
(from apache-beam==2.7.0.dev0) (0.19.7)
Requirement already satisfied: google-apitools<=0.5.20,>=0.5.18 in /home/jenkins/.local/lib/python2.7/site-packages
(from apache-beam==2.7.0.dev0) (0.5.20)
Requirement already satisfied: proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0 in /home/jenkins/.local/lib/python2.7/site-packages
(from apache-beam==2.7.0.dev0) (0.90.4)
Requirement already satisfied: googledatastore==7.0.1 in /home/jenkins/.local/lib/python2.7/site-packages
(from apache-beam==2.7.0.dev0) (7.0.1)
Requirement already satisfied: google-cloud-pubsub==0.26.0 in /home/jenkins/.local/lib/python2.7/site-packages
(from apache-beam==2.7.0.dev0) (0.26.0)
Requirement already satisfied: proto-google-cloud-pubsub-v1==0.15.4 in /home/jenkins/.local/lib/python2.7/site-packages
(from apache-beam==2.7.0.dev0) (0.15.4)
Requirement already satisfied: google-cloud-bigquery==0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages
(from apache-beam==2.7.0.dev0) (0.25.0)
Requirement already satisfied: nose>=1.3.7 in /home/jenkins/.local/lib/python2.7/site-packages
(from apache-beam==2.7.0.dev0) (1.3.7)
Collecting numpy<2,>=1.14.3 (from apache-beam==2.7.0.dev0)
  Using cached https://files.pythonhosted.org/packages/85/51/ba4564ded90e093dbb6adfc3e21f99ae953d9ad56477e1b0d4a93bacf7d3/numpy-1.15.0-cp27-cp27mu-manylinux1_x86_64.whl
Requirement already satisfied: pyhamcrest<2.0,>=1.9 in /home/jenkins/.local/lib/python2.7/site-packages
(from apache-beam==2.7.0.dev0) (1.9.0)
Requirement already satisfied: enum34>=1.0.4 in /usr/local/lib/python2.7/dist-packages
(from grpcio<2,>=1.8->apache-beam==2.7.0.dev0) (1.1.6)
Requirement already satisfied: docopt in /usr/local/lib/python2.7/dist-packages (from hdfs<3.0.0,>=2.1.0->apache-beam==2.7.0.dev0)
(0.6.2)
Requirement already satisfied: requests>=2.7.0 in /usr/local/lib/python2.7/dist-packages
(from hdfs<3.0.0,>=2.1.0->apache-beam==2.7.0.dev0) (2.18.4)
Requirement already satisfied: pbr>=0.11 in /usr/lib/python2.7/dist-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.7.0.dev0)
(1.8.0)
Requirement already satisfied: funcsigs>=1; python_version < "3.3" in /home/jenkins/.local/lib/python2.7/site-packages
(from mock<3.0.0,>=1.0.1->apache-beam==2.7.0.dev0) (1.0.2)
Requirement already satisfied: pyasn1>=0.1.7 in /usr/local/lib/python2.7/dist-packages
(from oauth2client<5,>=2.0.1->apache-beam==2.7.0.dev0) (0.4.2)
Requirement already satisfied: pyasn1-modules>=0.0.5 in /home/jenkins/.local/lib/python2.7/site-packages
(from oauth2client<5,>=2.0.1->apache-beam==2.7.0.dev0) (0.2.2)
Requirement already satisfied: rsa>=3.1.4 in /home/jenkins/.local/lib/python2.7/site-packages
(from oauth2client<5,>=2.0.1->apache-beam==2.7.0.dev0) (3.4.2)
Requirement already satisfied: setuptools in ./.beam_env/lib/python2.7/site-packages (from
protobuf<4,>=3.5.0.post1->apache-beam==2.7.0.dev0) (40.0.0)
Requirement already satisfied: pyparsing>=2.1.4 in /home/jenkins/.local/lib/python2.7/site-packages
(from pydot<1.3,>=1.2.0->apache-beam==2.7.0.dev0) (2.2.0)
Requirement already satisfied: fasteners>=0.14 in /home/jenkins/.local/lib/python2.7/site-packages
(from google-apitools<=0.5.20,>=0.5.18->apache-beam==2.7.0.dev0) (0.14.1)
Requirement already satisfied: googleapis-common-protos<2.0dev,>=1.5.2 in /home/jenkins/.local/lib/python2.7/site-packages
(from proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0->apache-beam==2.7.0.dev0)
(1.5.3)
Requirement already satisfied: gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0 in /home/jenkins/.local/lib/python2.7/site-packages
(from google-cloud-pubsub==0.26.0->apache-beam==2.7.0.dev0) (0.15.4)
Requirement already satisfied: google-cloud-core<0.26dev,>=0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages
(from google-cloud-pubsub==0.26.0->apache-beam==2.7.0.dev0) (0.25.0)
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python2.7/dist-packages
(from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.7.0.dev0) (2018.4.16)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /usr/local/lib/python2.7/dist-packages
(from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.7.0.dev0) (3.0.4)
Requirement already satisfied: idna<2.7,>=2.5 in /usr/local/lib/python2.7/dist-packages
(from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.7.0.dev0) (2.6)
Requirement already satisfied: urllib3<1.23,>=1.21.1 in /usr/local/lib/python2.7/dist-packages
(from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.7.0.dev0) (1.22)
Requirement already satisfied: monotonic>=0.1 in /home/jenkins/.local/lib/python2.7/site-packages
(from fasteners>=0.14->google-apitools<=0.5.20,>=0.5.18->apache-beam==2.7.0.dev0)
(1.5)
Requirement already satisfied: google-gax<0.16dev,>=0.15.7 in /home/jenkins/.local/lib/python2.7/site-packages
(from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.7.0.dev0)
(0.15.16)
Requirement already satisfied: grpc-google-iam-v1<0.12dev,>=0.11.1 in /home/jenkins/.local/lib/python2.7/site-packages
(from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.7.0.dev0)
(0.11.4)
Requirement already satisfied: google-auth-httplib2 in /home/jenkins/.local/lib/python2.7/site-packages
(from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.7.0.dev0)
(0.0.3)
Requirement already satisfied: google-auth<2.0.0dev,>=0.4.0 in /home/jenkins/.local/lib/python2.7/site-packages
(from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.7.0.dev0)
(1.5.0)
Requirement already satisfied: ply==3.8 in /home/jenkins/.local/lib/python2.7/site-packages
(from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.7.0.dev0)
(3.8)
Requirement already satisfied: cachetools>=2.0.0 in /home/jenkins/.local/lib/python2.7/site-packages
(from google-auth<2.0.0dev,>=0.4.0->google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.7.0.dev0)
(2.1.0)
Installing collected packages: numpy, apache-beam
  Found existing installation: numpy 1.13.3
    Not uninstalling numpy at /home/jenkins/.local/lib/python2.7/site-packages, outside environment
<https://builds.apache.org/job/beam_PerformanceTests_ParquetIOIT_HDFS/ws/.beam_env>
    Can't uninstall 'numpy'. No files were found to uninstall.
  Found existing installation: apache-beam 2.7.0.dev0
    Not uninstalling apache-beam at /home/jenkins/jenkins-slave/workspace/beam_PerformanceTests_TFRecordIOIT/src/sdks/python,
outside environment <https://builds.apache.org/job/beam_PerformanceTests_ParquetIOIT_HDFS/ws/.beam_env>
    Can't uninstall 'apache-beam'. No files were found to uninstall.
  Running setup.py develop for apache-beam
Successfully installed apache-beam numpy-1.15.0
[beam_PerformanceTests_ParquetIOIT_HDFS] $ /bin/bash -xe /tmp/jenkins6087940503217660596.sh
+ .beam_env/bin/python src/sdks/python/setup.py sdist --dist-dir=src/sdks/python/target
Traceback (most recent call last):
  File "src/sdks/python/setup.py", line 46, in <module>
    PACKAGE_VERSION = get_version()
  File "src/sdks/python/setup.py", line 41, in get_version
    exec(open(os.path.normpath('./apache_beam/version.py')).read(), global_names)  # pylint:
disable=exec-used
IOError: [Errno 2] No such file or directory: 'apache_beam/version.py'
Build step 'Execute shell' marked build as failure

Mime
View raw message