beam-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Apache Jenkins Server <jenk...@builds.apache.org>
Subject Build failed in Jenkins: beam_PerformanceTests_Python #71
Date Fri, 07 Jul 2017 00:21:19 GMT
See <https://builds.apache.org/job/beam_PerformanceTests_Python/71/display/redirect?page=changes>

Changes:

[klk] Add timeout to initialization of partition in KafkaIO

[klk] Ignore processing time timers in expired windows

[klk] Process timer firings for a window together

------------------------------------------
Started by timer
[EnvInject] - Loading node environment variables.
Building remotely on beam3 (beam) in workspace <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
 > git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
 > git config remote.origin.url https://github.com/apache/beam.git # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
 > git --version # timeout=10
 > git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/*
+refs/pull/${ghprbPullId}/*:refs/remotes/origin/pr/${ghprbPullId}/*
 > git rev-parse origin/master^{commit} # timeout=10
Checking out Revision 628dace9c173182299f322fa6ed974c75dbff4b5 (origin/master)
 > git config core.sparsecheckout # timeout=10
 > git checkout -f 628dace9c173182299f322fa6ed974c75dbff4b5
 > git rev-list 85a99e29448670ae6728a8ee2e4cd3ef95877c3e # timeout=10
Cleaning workspace
 > git rev-parse --verify HEAD # timeout=10
Resetting working tree
 > git reset --hard # timeout=10
 > git clean -fdx # timeout=10
[EnvInject] - Executing scripts and injecting environment variables after the SCM step.
[EnvInject] - Injecting as environment variables the properties content 
SPARK_LOCAL_IP=127.0.0.1

[EnvInject] - Variables injected successfully.
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/hudson4251429228094718244.sh
+ rm -rf PerfKitBenchmarker
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/hudson5841686889897044075.sh
+ git clone https://github.com/GoogleCloudPlatform/PerfKitBenchmarker.git
Cloning into 'PerfKitBenchmarker'...
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/hudson5139416760218626072.sh
+ pip install --user -r PerfKitBenchmarker/requirements.txt
Requirement already satisfied (use --upgrade to upgrade): python-gflags==3.1.1 in /home/jenkins/.local/lib/python2.7/site-packages
(from -r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied (use --upgrade to upgrade): jinja2>=2.7 in /usr/local/lib/python2.7/dist-packages
(from -r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied (use --upgrade to upgrade): setuptools in /usr/lib/python2.7/dist-packages
(from -r PerfKitBenchmarker/requirements.txt (line 16))
Requirement already satisfied (use --upgrade to upgrade): colorlog[windows]==2.6.0 in /home/jenkins/.local/lib/python2.7/site-packages
(from -r PerfKitBenchmarker/requirements.txt (line 17))
  Installing extra requirements: 'windows'
Requirement already satisfied (use --upgrade to upgrade): blinker>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages
(from -r PerfKitBenchmarker/requirements.txt (line 18))
Requirement already satisfied (use --upgrade to upgrade): futures>=3.0.3 in /home/jenkins/.local/lib/python2.7/site-packages
(from -r PerfKitBenchmarker/requirements.txt (line 19))
Requirement already satisfied (use --upgrade to upgrade): PyYAML==3.12 in /home/jenkins/.local/lib/python2.7/site-packages
(from -r PerfKitBenchmarker/requirements.txt (line 20))
Requirement already satisfied (use --upgrade to upgrade): pint>=0.7 in /home/jenkins/.local/lib/python2.7/site-packages
(from -r PerfKitBenchmarker/requirements.txt (line 21))
Requirement already satisfied (use --upgrade to upgrade): numpy in /home/jenkins/.local/lib/python2.7/site-packages
(from -r PerfKitBenchmarker/requirements.txt (line 22))
Requirement already satisfied (use --upgrade to upgrade): functools32 in /home/jenkins/.local/lib/python2.7/site-packages
(from -r PerfKitBenchmarker/requirements.txt (line 23))
Requirement already satisfied (use --upgrade to upgrade): contextlib2>=0.5.1 in /home/jenkins/.local/lib/python2.7/site-packages
(from -r PerfKitBenchmarker/requirements.txt (line 24))
Cleaning up...
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/hudson1618825673769026633.sh
+ pip install --user -e 'sdks/python/[gcp,test]'
Obtaining file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/sdks/python>
  Running setup.py (path:<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/sdks/python/setup.py)>
egg_info for package from file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/sdks/python>
    <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/sdks/python/setup.py>:66:
UserWarning: You are using version 1.5.4 of pip. However, version 7.0.0 is recommended.
      _PIP_VERSION, REQUIRED_PIP_VERSION
    no previously-included directories found matching 'doc/.build'
    
    Installed <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/sdks/python/nose-1.3.7-py2.7.egg>
    
    warning: no files found matching 'README.md'
    warning: no files found matching 'NOTICE'
    warning: no files found matching 'LICENSE'
  Installing extra requirements: 'gcp,test'
Requirement already satisfied (use --upgrade to upgrade): avro>=1.8.1,<2.0.0 in /home/jenkins/.local/lib/python2.7/site-packages
(from apache-beam==2.2.0.dev)
Requirement already satisfied (use --upgrade to upgrade): crcmod>=1.7,<2.0 in /home/jenkins/.local/lib/python2.7/site-packages
(from apache-beam==2.2.0.dev)
Requirement already satisfied (use --upgrade to upgrade): dill==0.2.6 in /home/jenkins/.local/lib/python2.7/site-packages
(from apache-beam==2.2.0.dev)
Requirement already satisfied (use --upgrade to upgrade): grpcio>=1.0,<2.0 in /home/jenkins/.local/lib/python2.7/site-packages
(from apache-beam==2.2.0.dev)
Requirement already satisfied (use --upgrade to upgrade): httplib2>=0.8,<0.10 in /home/jenkins/.local/lib/python2.7/site-packages
(from apache-beam==2.2.0.dev)
Requirement already satisfied (use --upgrade to upgrade): mock>=1.0.1,<3.0.0 in /home/jenkins/.local/lib/python2.7/site-packages
(from apache-beam==2.2.0.dev)
Requirement already satisfied (use --upgrade to upgrade): oauth2client>=2.0.1,<4.0.0
in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.2.0.dev)
Requirement already satisfied (use --upgrade to upgrade): protobuf>=3.2.0,<=3.3.0 in
/home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.2.0.dev)
Requirement already satisfied (use --upgrade to upgrade): pyyaml>=3.12,<4.0.0 in /home/jenkins/.local/lib/python2.7/site-packages
(from apache-beam==2.2.0.dev)
Requirement already satisfied (use --upgrade to upgrade): pyhamcrest>=1.9,<2.0 in /home/jenkins/.local/lib/python2.7/site-packages
(from apache-beam==2.2.0.dev)
Requirement already satisfied (use --upgrade to upgrade): six>=1.9 in /home/jenkins/.local/lib/python2.7/site-packages
(from apache-beam==2.2.0.dev)
Requirement already satisfied (use --upgrade to upgrade): google-apitools>=0.5.10,<=0.5.11
in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.2.0.dev)
Requirement already satisfied (use --upgrade to upgrade): proto-google-cloud-datastore-v1>=0.90.0,<=0.90.4
in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.2.0.dev)
Requirement already satisfied (use --upgrade to upgrade): googledatastore==7.0.1 in /home/jenkins/.local/lib/python2.7/site-packages
(from apache-beam==2.2.0.dev)
Requirement already satisfied (use --upgrade to upgrade): google-cloud-pubsub==0.26.0 in /home/jenkins/.local/lib/python2.7/site-packages
(from apache-beam==2.2.0.dev)
Requirement already satisfied (use --upgrade to upgrade): google-cloud-bigquery>=0.23.0,<0.26.0
in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.2.0.dev)
Requirement already satisfied (use --upgrade to upgrade): futures>=2.2.0 in /home/jenkins/.local/lib/python2.7/site-packages
(from grpcio>=1.0,<2.0->apache-beam==2.2.0.dev)
Requirement already satisfied (use --upgrade to upgrade): enum34>=1.0.4 in /home/jenkins/.local/lib/python2.7/site-packages
(from grpcio>=1.0,<2.0->apache-beam==2.2.0.dev)
Requirement already satisfied (use --upgrade to upgrade): pyasn1>=0.1.7 in /home/jenkins/.local/lib/python2.7/site-packages
(from oauth2client>=2.0.1,<4.0.0->apache-beam==2.2.0.dev)
Requirement already satisfied (use --upgrade to upgrade): pyasn1-modules>=0.0.5 in /home/jenkins/.local/lib/python2.7/site-packages
(from oauth2client>=2.0.1,<4.0.0->apache-beam==2.2.0.dev)
Requirement already satisfied (use --upgrade to upgrade): rsa>=3.1.4 in /home/jenkins/.local/lib/python2.7/site-packages
(from oauth2client>=2.0.1,<4.0.0->apache-beam==2.2.0.dev)
Requirement already satisfied (use --upgrade to upgrade): setuptools in /usr/lib/python2.7/dist-packages
(from protobuf>=3.2.0,<=3.3.0->apache-beam==2.2.0.dev)
Requirement already satisfied (use --upgrade to upgrade): googleapis-common-protos>=1.5.2,<2.0dev
in /home/jenkins/.local/lib/python2.7/site-packages (from proto-google-cloud-datastore-v1>=0.90.0,<=0.90.4->apache-beam==2.2.0.dev)
Installing collected packages: apache-beam
  Running setup.py develop for apache-beam
    <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/sdks/python/setup.py>:66:
UserWarning: You are using version 1.5.4 of pip. However, version 7.0.0 is recommended.
      _PIP_VERSION, REQUIRED_PIP_VERSION
    
    warning: no files found matching 'README.md'
    warning: no files found matching 'NOTICE'
    warning: no files found matching 'LICENSE'
    Creating /home/jenkins/.local/lib/python2.7/site-packages/apache-beam.egg-link (link to
.)
    apache-beam 2.2.0.dev is already the active version in easy-install.pth
    
    Installed <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/sdks/python>
Successfully installed apache-beam
Cleaning up...
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/hudson4219242185552476604.sh
+ python PerfKitBenchmarker/pkb.py --project=apache-beam-testing --dpb_log_level=INFO --maven_binary=/home/jenkins/tools/maven/latest/bin/mvn
--bigquery_table=beam_performance.pkb_results --official=true --beam_sdk=python --benchmarks=beam_integration_benchmark
--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
WARNING:root:File resource loader root perfkitbenchmarker/data/ycsb is not a directory.
2017-07-07 00:00:14,677 c8ba0976 MainThread INFO     Verbose logging to: /tmp/perfkitbenchmarker/runs/c8ba0976/pkb.log
2017-07-07 00:00:14,677 c8ba0976 MainThread INFO     PerfKitBenchmarker version: v1.12.0-55-g5eff150
2017-07-07 00:00:14,678 c8ba0976 MainThread INFO     Flag values:
--beam_sdk=python
--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
--maven_binary=/home/jenkins/tools/maven/latest/bin/mvn
--project=apache-beam-testing
--bigquery_table=beam_performance.pkb_results
--official
--dpb_log_level=INFO
--benchmarks=beam_integration_benchmark
2017-07-07 00:00:15,113 c8ba0976 MainThread WARNING  The key "flags" was not in the default
config, but was in user overrides. This may indicate a typo.
2017-07-07 00:00:15,222 c8ba0976 MainThread beam_integration_benchmark(1/1) INFO     Provisioning
resources for benchmark beam_integration_benchmark
2017-07-07 00:00:15,224 c8ba0976 MainThread beam_integration_benchmark(1/1) INFO     Preparing
benchmark beam_integration_benchmark
2017-07-07 00:00:15,225 c8ba0976 MainThread beam_integration_benchmark(1/1) INFO     Running:
git clone https://github.com/apache/beam.git
2017-07-07 00:00:19,890 c8ba0976 MainThread beam_integration_benchmark(1/1) INFO     Running:
/home/jenkins/tools/maven/latest/bin/mvn clean install -DskipTests -Dcheckstyle.skip=true
-Pdataflow-runner
2017-07-07 00:11:18,665 c8ba0976 MainThread beam_integration_benchmark(1/1) INFO     Running
benchmark beam_integration_benchmark
2017-07-07 00:11:18,674 c8ba0976 MainThread beam_integration_benchmark(1/1) INFO     Running:
python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing"
"--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it"
"--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner
--sdk_location=/tmp/perfkitbenchmarker/runs/c8ba0976/beam/sdks/python/target/apache-beam-2.2.0.dev.tar.gz
2017-07-07 00:21:18,677 c8ba0976 Thread-4 ERROR    IssueCommand timed out after 600 seconds.
Killing command "python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test
--attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it"
"--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output"
--runner=TestDataflowRunner --sdk_location=/tmp/perfkitbenchmarker/runs/c8ba0976/beam/sdks/python/target/apache-beam-2.2.0.dev.tar.gz".
2017-07-07 00:21:18,680 c8ba0976 MainThread beam_integration_benchmark(1/1) INFO     Ran python
setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing"
"--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it"
"--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner
--sdk_location=/tmp/perfkitbenchmarker/runs/c8ba0976/beam/sdks/python/target/apache-beam-2.2.0.dev.tar.gz.
Got return code (-9).
STDOUT: running nosetests
running egg_info
writing requirements to apache_beam.egg-info/requires.txt
writing apache_beam.egg-info/PKG-INFO
writing top-level names to apache_beam.egg-info/top_level.txt
writing dependency_links to apache_beam.egg-info/dependency_links.txt
writing entry points to apache_beam.egg-info/entry_points.txt
reading manifest file 'apache_beam.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
writing manifest file 'apache_beam.egg-info/SOURCES.txt'

STDERR: setup.py:66: UserWarning: You are using version 1.5.4 of pip. However, version 7.0.0
is recommended.
  _PIP_VERSION, REQUIRED_PIP_VERSION
test_wordcount_it (apache_beam.examples.wordcount_it_test.WordCountIT) ... /tmp/perfkitbenchmarker/runs/c8ba0976/beam/sdks/python/apache_beam/io/gcp/gcsio.py:121:
DeprecationWarning: object() takes no parameters
  super(GcsIO, cls).__new__(cls, storage_client))

2017-07-07 00:21:18,681 c8ba0976 MainThread beam_integration_benchmark(1/1) ERROR    Error
during benchmark beam_integration_benchmark
Traceback (most recent call last):
  File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",>
line 559, in RunBenchmark
    DoRunPhase(spec, collector, detailed_timer)
  File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",>
line 472, in DoRunPhase
    samples = spec.BenchmarkRun(spec)
  File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",>
line 128, in Run
    job_type=job_type)
  File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",>
line 79, in SubmitJob
    assert retcode == 0, "Integration Test Failed."
AssertionError: Integration Test Failed.
2017-07-07 00:21:18,683 c8ba0976 MainThread beam_integration_benchmark(1/1) ERROR    Benchmark
1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2017-07-07 00:21:18,702 c8ba0976 MainThread INFO     Benchmark run statuses:
---------------------------------------------------------------
Name                        UID                          Status
---------------------------------------------------------------
beam_integration_benchmark  beam_integration_benchmark0  FAILED
---------------------------------------------------------------
Success rate: 0.00% (0/1)
2017-07-07 00:21:18,703 c8ba0976 MainThread INFO     Complete logs can be found at: /tmp/perfkitbenchmarker/runs/c8ba0976/pkb.log
2017-07-07 00:21:18,703 c8ba0976 MainThread INFO     Completion statuses can be found at:
/tmp/perfkitbenchmarker/runs/c8ba0976/completion_statuses.json
Build step 'Execute shell' marked build as failure
Not sending mail to unregistered user klk@google.com

Mime
View raw message