beam-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Apache Jenkins Server <jenk...@builds.apache.org>
Subject Build failed in Jenkins: beam_Dependency_Check #153
Date Mon, 01 Oct 2018 12:06:51 GMT
See <https://builds.apache.org/job/beam_Dependency_Check/153/display/redirect?page=changes>

Changes:

[pablo] Adding display data to BQ write transform

[qinyeli] Interactive Beam -- read_cache_ids and write_cache_ids

[qinyeli] Interactive Beam -- renaming variables and functions

[afulton] Fix unstable test by ensuring acks are processed synchronously

[andrew] Ensuring Spotless Java compliance

[qinyeli] Interactive Beam -- fixing PTransform # display issue

[jeff] Composite transform lambdas via PTransform::compose

[pablo] Updating BigQuerySink pydoc

[huangry] Refactor CombineGroupValues by using existing function in CombineFn.

[github] typo Python -> Go

[mxm] Add tests for port supplier methods in ServerFactory

[ptomasroos] Fix combine.go to run on dataflow

[github] Update TODO to reference Jira

[github] Fix sliding time windows example documentation

[github] Update session windows Python to match description

[aaltay] [BEAM-5209] Precommit timings notebook: fix illegal characters in URL,

[shaun] added avroio package

[pablo] [BEAM-5240] Add metrics dashboard deployment script and logic (#6277)

[aaltay] Remove standard_library.import_aliases() calls (#6471)

[github] Fix sliding time windows example documentation

[pablo] Revert "[BEAM-5240] Add metrics dashboard deployment script and logic

[mergebot] Fix a few typos in beam site docs

[shaun] updated read emits to support both string and custom type reflects

[apilloud] [BEAM-5491] Update NightlySnapshot job name

[github] [BEAM-5378] Remove go wordcap example Dataflow runner (#6395)

[ehudm] GCSIO: Allow empty object prefix in list_prefix().

[shaun] added avro write support

[github] Trigger pre-commit checks

[swegner] Upgrade assertj dependecny to latest (3.11.1)

[github] Improved Flatten input error message

[mergebot] [BEAM-5436] Improve docs for Go SDK

[github] Undo changes to website programming guide :rewind:

[migryz] Add metrics dashboard deployment script and logic

[migryz] Fix rat issues

[aaltay] [BEAM-1251] Upgrade pylint version for py27-lint3 (#6489)

[amaliujia] [BEAM-5506] Add reference link.

[aaltay] [BEAM-5319] Partially port runners (#6451)

[valentyn] Fix futurize-based checker in py27-lint3 compatibility tox test suite.

[ehudm] Pass --pubsubRooturl option to Dataflow runner.

[pablo] Fixing Py2-3 lint issue

[github] fix typo

[ccy] Revert "[BEAM-4747] mkdirs if they don't exist in localfilesystem

[ankurgoenka] Add PortablePipelineOption for environment

[ankurgoenka] Lazy initializing JobBundleFactory supporting multiple type of

[shaun] updated to be in-line with beam project specifications

[shaun] update package log prints

[shaun] added readavro example

[shaun] updated example package header

[shaun] removed output.avro file

[kedin] Fix Java11 Jira link

[github] Adding clarification to use WriteToBigQuery

[amyrvold] Website publiish task and Jenkins job

[yifanzou] [BEAM-5339] Modify the dependency tool based on new JIRA policy.

[ankurgoenka] Adding environment to python portable options

[ankurgoenka] Go Changes for using configurable environment. Only docker is supported

[github] Fix trailing whitespace

[daniel.o.programmer] [BEAM-5304] Adding ReferenceRunner Job Server Gradle subproject.

[robertwb] [BEAM-5270] Fix ToString coder to return bytes objects in Python 3.

[kevinsi] When getting display data from a runtime parameter, don't call get().

[kevinsi] Randomize the reduced splits in BigtableIO so that multiple workers may

[scott] [BEAM-5518] Ignore failing ssl validation of globenewswire (#6502)

[pablo] Updating Dataflow API protocol buffers

[altay] update dataflow container name

[thw] [BEAM-5509] Fix int/long pipeline option serialization in

[robertwb] [BEAM-5500] Fix memory leak in pickler.

[thw] [BEAM-5443] Pipeline option defaults for portable runner.

[scott] [BEAM-4496] Fix branch fetch failure on job_PostCommit_Website_Publish

------------------------------------------
[...truncated 487.05 KB...]
if ! [ -z "${PYTHONHOME+_}" ] ; then
    _OLD_VIRTUAL_PYTHONHOME="$PYTHONHOME"
    unset PYTHONHOME
fi

if [ -z "${VIRTUAL_ENV_DISABLE_PROMPT-}" ] ; then
    _OLD_VIRTUAL_PS1="$PS1"
    if [ "x" != x ] ; then
        PS1="$PS1"
    else
        PS1="(`basename \"$VIRTUAL_ENV\"`) $PS1"
    fi
    export PS1
fi
basename "$VIRTUAL_ENV"

# Make sure to unalias pydoc if it's already there
alias pydoc 2>/dev/null >/dev/null && unalias pydoc

pydoc () {
    python -m pydoc "$@"
}

# This should detect bash and zsh, which have a hash command that must
# be called to get it to forget past commands.  Without forgetting
# past commands the $PATH changes we made may not be respected
if [ -n "${BASH-}" ] || [ -n "${ZSH_VERSION-}" ] ; then
    hash -r 2>/dev/null
fi
pip install --upgrade google-cloud-bigquery
Collecting google-cloud-bigquery
  Using cached https://files.pythonhosted.org/packages/27/0b/4b8d48d602e12766187203767ebcf213f9e8297e1e1a38b714d5f921d5f6/google_cloud_bigquery-1.5.1-py2.py3-none-any.whl
Collecting google-api-core<2.0.0dev,>=1.0.0 (from google-cloud-bigquery)
  Downloading https://files.pythonhosted.org/packages/a5/be/de30100034c391f4c56e2543f1507eb1b30b3030bd9a6764dd6cfe7a954e/google_api_core-1.4.1-py2.py3-none-any.whl
(53kB)
Collecting google-cloud-core<0.29dev,>=0.28.0 (from google-cloud-bigquery)
  Using cached https://files.pythonhosted.org/packages/0f/41/ae2418b4003a14cf21c1c46d61d1b044bf02cf0f8f91598af572b9216515/google_cloud_core-0.28.1-py2.py3-none-any.whl
Collecting google-resumable-media>=0.2.1 (from google-cloud-bigquery)
  Using cached https://files.pythonhosted.org/packages/77/95/2e4020a54366423ddba715f89fb7ca456c8f048b15cada6cd6a54cf10e8c/google_resumable_media-0.3.1-py2.py3-none-any.whl
Collecting protobuf>=3.4.0 (from google-api-core<2.0.0dev,>=1.0.0->google-cloud-bigquery)
  Using cached https://files.pythonhosted.org/packages/b8/c2/b7f587c0aaf8bf2201405e8162323037fe8d17aa21d3c7dda811b8d01469/protobuf-3.6.1-cp27-cp27mu-manylinux1_x86_64.whl
Requirement already satisfied, skipping upgrade: setuptools>=34.0.0 in ./dependency/check/lib/python2.7/site-packages
(from google-api-core<2.0.0dev,>=1.0.0->google-cloud-bigquery) (40.4.3)
Collecting six>=1.10.0 (from google-api-core<2.0.0dev,>=1.0.0->google-cloud-bigquery)
  Using cached https://files.pythonhosted.org/packages/67/4b/141a581104b1f6397bfa78ac9d43d8ad29a7ca43ea90a2d863fe3056e86a/six-1.11.0-py2.py3-none-any.whl
Collecting google-auth<2.0.0dev,>=0.4.0 (from google-api-core<2.0.0dev,>=1.0.0->google-cloud-bigquery)
  Using cached https://files.pythonhosted.org/packages/58/cb/96dbb4e50e7a9d856e89cc9c8e36ab1055f9774f7d85f37e2156c1d79d9f/google_auth-1.5.1-py2.py3-none-any.whl
Collecting futures>=3.2.0; python_version < "3.2" (from google-api-core<2.0.0dev,>=1.0.0->google-cloud-bigquery)
  Using cached https://files.pythonhosted.org/packages/2d/99/b2c4e9d5a30f6471e410a146232b4118e697fa3ffc06d6a65efde84debd0/futures-3.2.0-py2-none-any.whl
Collecting pytz (from google-api-core<2.0.0dev,>=1.0.0->google-cloud-bigquery)
  Using cached https://files.pythonhosted.org/packages/30/4e/27c34b62430286c6d59177a0842ed90dc789ce5d1ed740887653b898779a/pytz-2018.5-py2.py3-none-any.whl
Collecting googleapis-common-protos<2.0dev,>=1.5.3 (from google-api-core<2.0.0dev,>=1.0.0->google-cloud-bigquery)
Collecting requests<3.0.0dev,>=2.18.0 (from google-api-core<2.0.0dev,>=1.0.0->google-cloud-bigquery)
  Using cached https://files.pythonhosted.org/packages/65/47/7e02164a2a3db50ed6d8a6ab1d6d60b69c4c3fdf57a284257925dfc12bda/requests-2.19.1-py2.py3-none-any.whl
Collecting rsa>=3.1.4 (from google-auth<2.0.0dev,>=0.4.0->google-api-core<2.0.0dev,>=1.0.0->google-cloud-bigquery)
  Using cached https://files.pythonhosted.org/packages/02/e5/38518af393f7c214357079ce67a317307936896e961e35450b70fad2a9cf/rsa-4.0-py2.py3-none-any.whl
Collecting cachetools>=2.0.0 (from google-auth<2.0.0dev,>=0.4.0->google-api-core<2.0.0dev,>=1.0.0->google-cloud-bigquery)
  Using cached https://files.pythonhosted.org/packages/0a/58/cbee863250b31d80f47401d04f34038db6766f95dea1cc909ea099c7e571/cachetools-2.1.0-py2.py3-none-any.whl
Collecting pyasn1-modules>=0.2.1 (from google-auth<2.0.0dev,>=0.4.0->google-api-core<2.0.0dev,>=1.0.0->google-cloud-bigquery)
  Using cached https://files.pythonhosted.org/packages/19/02/fa63f7ba30a0d7b925ca29d034510fc1ffde53264b71b4155022ddf3ab5d/pyasn1_modules-0.2.2-py2.py3-none-any.whl
Collecting certifi>=2017.4.17 (from requests<3.0.0dev,>=2.18.0->google-api-core<2.0.0dev,>=1.0.0->google-cloud-bigquery)
  Using cached https://files.pythonhosted.org/packages/df/f7/04fee6ac349e915b82171f8e23cee63644d83663b34c539f7a09aed18f9e/certifi-2018.8.24-py2.py3-none-any.whl
Collecting chardet<3.1.0,>=3.0.2 (from requests<3.0.0dev,>=2.18.0->google-api-core<2.0.0dev,>=1.0.0->google-cloud-bigquery)
  Using cached https://files.pythonhosted.org/packages/bc/a9/01ffebfb562e4274b6487b4bb1ddec7ca55ec7510b22e4c51f14098443b8/chardet-3.0.4-py2.py3-none-any.whl
Collecting urllib3<1.24,>=1.21.1 (from requests<3.0.0dev,>=2.18.0->google-api-core<2.0.0dev,>=1.0.0->google-cloud-bigquery)
  Using cached https://files.pythonhosted.org/packages/bd/c9/6fdd990019071a4a32a5e7cb78a1d92c53851ef4f56f62a3486e6a7d8ffb/urllib3-1.23-py2.py3-none-any.whl
Collecting idna<2.8,>=2.5 (from requests<3.0.0dev,>=2.18.0->google-api-core<2.0.0dev,>=1.0.0->google-cloud-bigquery)
  Using cached https://files.pythonhosted.org/packages/4b/2a/0276479a4b3caeb8a8c1af2f8e4355746a97fab05a372e4a2c6a6b876165/idna-2.7-py2.py3-none-any.whl
Collecting pyasn1>=0.1.3 (from rsa>=3.1.4->google-auth<2.0.0dev,>=0.4.0->google-api-core<2.0.0dev,>=1.0.0->google-cloud-bigquery)
  Using cached https://files.pythonhosted.org/packages/d1/a1/7790cc85db38daa874f6a2e6308131b9953feb1367f2ae2d1123bb93a9f5/pyasn1-0.4.4-py2.py3-none-any.whl
Installing collected packages: six, protobuf, pyasn1, rsa, cachetools, pyasn1-modules, google-auth,
futures, pytz, googleapis-common-protos, certifi, chardet, urllib3, idna, requests, google-api-core,
google-cloud-core, google-resumable-media, google-cloud-bigquery
Successfully installed cachetools-2.1.0 certifi-2018.8.24 chardet-3.0.4 futures-3.2.0 google-api-core-1.4.1
google-auth-1.5.1 google-cloud-bigquery-1.5.1 google-cloud-core-0.28.1 google-resumable-media-0.3.1
googleapis-common-protos-1.5.3 idna-2.7 protobuf-3.6.1 pyasn1-0.4.4 pyasn1-modules-0.2.2 pytz-2018.5
requests-2.19.1 rsa-4.0 six-1.11.0 urllib3-1.23
rm -f build/dependencyUpdates/beam-dependency-check-report.txt

# Insall packages and run the unit tests of the report generator and the jira manager
pip install mock jira pyyaml
Collecting mock
  Using cached https://files.pythonhosted.org/packages/e6/35/f187bdf23be87092bd0f1200d43d23076cee4d0dec109f195173fd3ebc79/mock-2.0.0-py2.py3-none-any.whl
Collecting jira
  Using cached https://files.pythonhosted.org/packages/cb/c2/4203b6cad72d115b0ad62ae7028567b538c8933745f665d0eee5082baf67/jira-2.0.0-py2.py3-none-any.whl
Collecting pyyaml
Collecting pbr>=0.11 (from mock)
  Using cached https://files.pythonhosted.org/packages/69/1c/98cba002ed975a91a0294863d9c774cc0ebe38e05bbb65e83314550b1677/pbr-4.2.0-py2.py3-none-any.whl
Collecting funcsigs>=1; python_version < "3.3" (from mock)
  Using cached https://files.pythonhosted.org/packages/69/cb/f5be453359271714c01b9bd06126eaf2e368f1fddfff30818754b5ac2328/funcsigs-1.0.2-py2.py3-none-any.whl
Requirement already satisfied: six>=1.9 in ./dependency/check/lib/python2.7/site-packages
(from mock) (1.11.0)
Requirement already satisfied: requests>=2.10.0 in ./dependency/check/lib/python2.7/site-packages
(from jira) (2.19.1)
Collecting oauthlib[signedtoken]>=1.0.0 (from jira)
  Using cached https://files.pythonhosted.org/packages/e6/d1/ddd9cfea3e736399b97ded5c2dd62d1322adef4a72d816f1ed1049d6a179/oauthlib-2.1.0-py2.py3-none-any.whl
Collecting defusedxml (from jira)
  Using cached https://files.pythonhosted.org/packages/87/1c/17f3e3935a913dfe2a5ca85fa5ccbef366bfd82eb318b1f75dadbf0affca/defusedxml-0.5.0-py2.py3-none-any.whl
Collecting requests-toolbelt (from jira)
  Using cached https://files.pythonhosted.org/packages/97/8a/d710f792d6f6ecc089c5e55b66e66c3f2f35516a1ede5a8f54c13350ffb0/requests_toolbelt-0.8.0-py2.py3-none-any.whl
Collecting requests-oauthlib>=0.6.1 (from jira)
  Using cached https://files.pythonhosted.org/packages/94/e7/c250d122992e1561690d9c0f7856dadb79d61fd4bdd0e598087dce607f6c/requests_oauthlib-1.0.0-py2.py3-none-any.whl
Requirement already satisfied: setuptools>=20.10.1 in ./dependency/check/lib/python2.7/site-packages
(from jira) (40.4.3)
Requirement already satisfied: certifi>=2017.4.17 in ./dependency/check/lib/python2.7/site-packages
(from requests>=2.10.0->jira) (2018.8.24)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in ./dependency/check/lib/python2.7/site-packages
(from requests>=2.10.0->jira) (3.0.4)
Requirement already satisfied: urllib3<1.24,>=1.21.1 in ./dependency/check/lib/python2.7/site-packages
(from requests>=2.10.0->jira) (1.23)
Requirement already satisfied: idna<2.8,>=2.5 in ./dependency/check/lib/python2.7/site-packages
(from requests>=2.10.0->jira) (2.7)
Collecting cryptography; extra == "signedtoken" (from oauthlib[signedtoken]>=1.0.0->jira)
  Using cached https://files.pythonhosted.org/packages/87/e6/915a482dbfef98bbdce6be1e31825f591fc67038d4ee09864c1d2c3db371/cryptography-2.3.1-cp27-cp27mu-manylinux1_x86_64.whl
Collecting pyjwt>=1.0.0; extra == "signedtoken" (from oauthlib[signedtoken]>=1.0.0->jira)
  Using cached https://files.pythonhosted.org/packages/93/d1/3378cc8184a6524dc92993090ee8b4c03847c567e298305d6cf86987e005/PyJWT-1.6.4-py2.py3-none-any.whl
Collecting enum34; python_version < "3" (from cryptography; extra == "signedtoken"->oauthlib[signedtoken]>=1.0.0->jira)
  Using cached https://files.pythonhosted.org/packages/c5/db/e56e6b4bbac7c4a06de1c50de6fe1ef3810018ae11732a50f15f62c7d050/enum34-1.1.6-py2-none-any.whl
Collecting asn1crypto>=0.21.0 (from cryptography; extra == "signedtoken"->oauthlib[signedtoken]>=1.0.0->jira)
  Using cached https://files.pythonhosted.org/packages/ea/cd/35485615f45f30a510576f1a56d1e0a7ad7bd8ab5ed7cdc600ef7cd06222/asn1crypto-0.24.0-py2.py3-none-any.whl
Collecting cffi!=1.11.3,>=1.7 (from cryptography; extra == "signedtoken"->oauthlib[signedtoken]>=1.0.0->jira)
  Using cached https://files.pythonhosted.org/packages/14/dd/3e7a1e1280e7d767bd3fa15791759c91ec19058ebe31217fe66f3e9a8c49/cffi-1.11.5-cp27-cp27mu-manylinux1_x86_64.whl
Collecting ipaddress; python_version < "3" (from cryptography; extra == "signedtoken"->oauthlib[signedtoken]>=1.0.0->jira)
  Using cached https://files.pythonhosted.org/packages/fc/d0/7fc3a811e011d4b388be48a0e381db8d990042df54aa4ef4599a31d39853/ipaddress-1.0.22-py2.py3-none-any.whl
Collecting pycparser (from cffi!=1.11.3,>=1.7->cryptography; extra == "signedtoken"->oauthlib[signedtoken]>=1.0.0->jira)
Installing collected packages: pbr, funcsigs, mock, enum34, asn1crypto, pycparser, cffi, ipaddress,
cryptography, pyjwt, oauthlib, defusedxml, requests-toolbelt, requests-oauthlib, jira, pyyaml
Successfully installed asn1crypto-0.24.0 cffi-1.11.5 cryptography-2.3.1 defusedxml-0.5.0 enum34-1.1.6
funcsigs-1.0.2 ipaddress-1.0.22 jira-2.0.0 mock-2.0.0 oauthlib-2.1.0 pbr-4.2.0 pycparser-2.19
pyjwt-1.6.4 pyyaml-3.13 requests-oauthlib-1.0.0 requests-toolbelt-0.8.0
cd $WORKSPACE/src/.test-infra/jenkins
python -m dependency_check.dependency_check_report_generator_test
INFO:root:

Start processing:  - group1:artifact1 [Release1-123 -> Release2-456]
.INFO:root:

Start processing:  - group1:artifact1 [0.rc1.0 -> 0.rc2.0]
.INFO:root:Clean Up Starts
.INFO:root:

Start processing: - group1:artifact1 (1.0.0, 2.0.0)
Traceback (most recent call last):
  File "dependency_check/dependency_check_report_generator.py", line 107, in prioritize_dependencies
  File "dependency_check/dependency_check_report_generator.py", line 76, in extract_single_dep
InvalidFormatError: Failed to extract the dependency information: - group1:artifact1 (1.0.0,
2.0.0)
INFO:root:

Start processing:  - group2:artifact2 [1.0.0 -> 2.0.0]
.INFO:root:

Start processing:  - group1:artifact1 [1.0.0 -> 3.0.0]
INFO:root:

Start processing:  - group2:artifact2 [1.0.0 -> 1.3.0]
INFO:root:

Start processing:  - group3:artifact3 [1.0.0 -> 1.1.0]
INFO:root:

Start processing:  - group4:artifact4 [1.0.0 -> 1.1.0]
.
----------------------------------------------------------------------
Ran 5 tests in 0.028s

OK


Test : test_dep_with_nondigit_major_versions


Test : test_dep_with_nondigit_minor_versions


Test : test_empty_dep_input


Test : test_invalid_dep_input


Test : test_normal_dep_input
python -m jira_utils.jira_manager_test
INFO:root:Start handling the JIRA issues for Python dependency: dep0 1.0.0
INFO:root:Creating a new JIRA issue to track dep0 upgrade process
INFO:root:Found owners of dep0: owner0, owner1 , owner2,
INFO:root:Created a new issue <MagicMock name='JiraClient().create_issue().key' id='139931617705552'>
of dep0 1.0.0
.INFO:root:Start handling the JIRA issues for Java dependency: group0:artifact0 1.0.0
INFO:root:Found the parent issue BEAM-1000. Continuous to create or update the sub-task for
group0:artifact0
INFO:root:Creating a new JIRA issue to track group0:artifact0 upgrade process
INFO:root:Found owners of group0:artifact0: owner0
INFO:root:Created a new issue <MagicMock name='JiraClient().create_issue().key' id='139931617634832'>
of group0:artifact0 1.0.0
.INFO:root:Start handling the JIRA issues for Java dependency: group0:artifact0 1.0.0
INFO:root:The parent issue BEAM-1000 is not opening. Attempt reopening the issue
INFO:root:Found the parent issue BEAM-1000. Continuous to create or update the sub-task for
group0:artifact0
INFO:root:Creating a new JIRA issue to track group0:artifact0 upgrade process
INFO:root:Found owners of group0:artifact0: owner0
INFO:root:Created a new issue <MagicMock name='JiraClient().create_issue().key' id='139931617128720'>
of group0:artifact0 1.0.0
.INFO:root:Start handling the JIRA issues for Python dependency: dep0 1.0.0
INFO:root:Updating JIRA issue BEAM-1000 to track dep0 upgrade process
WARNING:root:Could not find owners for dep0
INFO:root:Reopened the issue BEAM-1000 for dep0 1.0.0
.INFO:root:Start handling the JIRA issues for Python dependency: dep0 1.3
INFO:root:Updating JIRA issue BEAM-1000 to track dep0 upgrade process
WARNING:root:Could not find owners for dep0
INFO:root:Reopened the issue BEAM-1000 for dep0 1.3
.INFO:root:Start handling the JIRA issues for Python dependency: dep0 1.0.0
INFO:root:Updating JIRA issue BEAM-1000 to track dep0 upgrade process
WARNING:root:Could not find owners for dep0
INFO:root:Updated the existing issue BEAM-1000 of dep0 1.0.0
.
----------------------------------------------------------------------
Ran 6 tests in 0.028s

OK


Test : test_run_with_creating_new_issue


Test : test_run_with_creating_new_subtask


Test : test_run_with_reopening_existing_parent_issue


Test : test_run_with_reopening_issue_with_fixversions


Test : test_run_with_reopening_issue_with_new_release_available


Test : test_run_with_updating_existing_task
python -m dependency_check.version_comparer_test
.....
----------------------------------------------------------------------
Ran 5 tests in 0.000s

OK


Test : test_compare_major_verison_true


Test : test_compare_minor_version_false


Test : test_compare_minor_version_true


Test : test_compare_non_semantic_version_true


Test : test_compare_same_version_false

echo "<html><body>" > $WORKSPACE/src/build/dependencyUpdates/beam-dependency-check-report.html

python -m dependency_check/dependency_check_report_generator Python
Traceback (most recent call last):
  File "/usr/lib/python2.7/runpy.py", line 174, in _run_module_as_main
    "__main__", fname, loader, pkg_name)
  File "/usr/lib/python2.7/runpy.py", line 72, in _run_code
    exec code in run_globals
  File "<https://builds.apache.org/job/beam_Dependency_Check/ws/src/.test-infra/jenkins/dependency_check/dependency_check_report_generator.py",>
line 25, in <module>
    import version_comparer
ImportError: No module named version_comparer
Build step 'Execute shell' marked build as failure

Mime
View raw message