beam-builds mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Apache Jenkins Server <jenk...@builds.apache.org>
Subject Build failed in Jenkins: beam_PostCommit_XVR_Spark #947
Date Wed, 10 Jun 2020 18:31:59 GMT
See <https://builds.apache.org/job/beam_PostCommit_XVR_Spark/947/display/redirect?page=changes>

Changes:

[klk] Split Nexmark QueryTest and SqlQueryTest for clarity

[klk] Add ZetaSQL Nexmark variant

[annaqin] [BEAM-10225] Add log message when starting job server

[valentyn] [BEAM-10227] Switches typing version modifier to python_full_version so

[kamil.wasilewski] [BEAM-8134] Grafana dashboards for Nexmark tests


------------------------------------------
[...truncated 798.42 KB...]
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'sdk_worker_parallelism'
was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'dataflow_kms_key' was
already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'enable_streaming_engine'
was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'project' was already
added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_region' was already
added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_zone' was already
added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'files_to_stage' was already
added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'streaming' was already
added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'experiments' was already
added
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STOPPED
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STARTING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to RUNNING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
ok
test_project (apache_beam.transforms.sql_test.SqlTransformTest) ... INFO:apache_beam.utils.subprocess_server:Using
pre-built snapshot at <https://builds.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/sdks/java/extensions/sql/expansion-service/build/libs/beam-sdks-java-extensions-sql-expansion-service-2.23.0-SNAPSHOT.jar>
INFO:apache_beam.utils.subprocess_server:Starting service with ['java' '-jar' '<https://builds.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/sdks/java/extensions/sql/expansion-service/build/libs/beam-sdks-java-extensions-sql-expansion-service-2.23.0-SNAPSHOT.jar'>
'38353']
DEBUG:root:Waiting for grpc channel to be ready at localhost:38353.
INFO:apache_beam.utils.subprocess_server:Starting expansion service at localhost:38353
DEBUG:root:Waiting for grpc channel to be ready at localhost:38353.
INFO:apache_beam.utils.subprocess_server:	beam:external:java:sql:v1: org.apache.beam.sdk.expansion.service.ExpansionService$ExternalTransformRegistrarLoader$$Lambda$1/1327763628@593634ad
INFO:apache_beam.utils.subprocess_server:	beam:external:java:generate_sequence:v1: org.apache.beam.sdk.expansion.service.ExpansionService$ExternalTransformRegistrarLoader$$Lambda$1/1327763628@20fa23c1
DEBUG:root:Waiting for grpc channel to be ready at localhost:38353.
DEBUG:root:Waiting for grpc channel to be ready at localhost:38353.
DEBUG:root:Waiting for grpc channel to be ready at localhost:38353.
DEBUG:root:Waiting for grpc channel to be ready at localhost:38353.
INFO:apache_beam.utils.subprocess_server:Jun 10, 2020 6:30:48 PM org.apache.beam.sdk.expansion.service.ExpansionService
expand
INFO:apache_beam.utils.subprocess_server:INFO: Expanding 'SqlTransform(beam:external:java:sql:v1)'
with URN 'beam:external:java:sql:v1'
INFO:apache_beam.utils.subprocess_server:Jun 10, 2020 6:30:50 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner
convertToBeamRel
INFO:apache_beam.utils.subprocess_server:INFO: SQL:
INFO:apache_beam.utils.subprocess_server:SELECT `PCOLLECTION`.`id`, `PCOLLECTION`.`flt`
INFO:apache_beam.utils.subprocess_server:FROM `beam`.`PCOLLECTION` AS `PCOLLECTION`
INFO:apache_beam.utils.subprocess_server:Jun 10, 2020 6:30:50 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner
convertToBeamRel
INFO:apache_beam.utils.subprocess_server:INFO: SQLPlan>
INFO:apache_beam.utils.subprocess_server:LogicalProject(id=[$0], flt=[$2])
INFO:apache_beam.utils.subprocess_server:  BeamIOSourceRel(table=[[beam, PCOLLECTION]])
INFO:apache_beam.utils.subprocess_server:
INFO:apache_beam.utils.subprocess_server:Jun 10, 2020 6:30:50 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner
convertToBeamRel
INFO:apache_beam.utils.subprocess_server:INFO: BEAMPlan>
INFO:apache_beam.utils.subprocess_server:BeamCalcRel(expr#0..2=[{inputs}], id=[$t0], flt=[$t2])
INFO:apache_beam.utils.subprocess_server:  BeamIOSourceRel(table=[[beam, PCOLLECTION]])
INFO:apache_beam.utils.subprocess_server:
DEBUG:root:Sending SIGINT to job_server
WARNING:root:Make sure that locally built Python SDK docker image has Python 2.7 interpreter.
INFO:root:Using Python SDK docker image: apache/beam_python2.7_sdk:2.23.0.dev. If the image
is not available at local, we will try to pull from hub.docker.com
WARNING:root:Make sure that locally built Python SDK docker image has Python 2.7 interpreter.
INFO:root:Using Python SDK docker image: apache/beam_python2.7_sdk:2.23.0.dev. If the image
is not available at local, we will try to pull from hub.docker.com
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function
lift_combiners at 0x7f91e7655488> ====================
DEBUG:apache_beam.runners.portability.fn_api_runner.translations:17 [1, 1, 1, 1, 1, 1, 1,
1, 1, 1, 1, 1, 1, 1, 1, 1, 1]
DEBUG:apache_beam.runners.portability.fn_api_runner.translations:Stages: ['ref_AppliedPTransform_Create/Impulse_3\n
 Create/Impulse:beam:transform:impulse:v1\n  must follow: \n  downstream_side_inputs: <unknown>',
'ref_AppliedPTransform_Create/FlatMap(<lambda at core.py:2623>)_4\n  Create/FlatMap(<lambda
at core.py:2623>):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>',
'ref_AppliedPTransform_Create/Map(decode)_6\n  Create/Map(decode):beam:transform:pardo:v1\n
 must follow: \n  downstream_side_inputs: <unknown>', 'external_4SqlTransform(beam:external:java:sql:v1)/BeamIOSourceRel_2/Convert.ConvertTransform/ParDo(Anonymous)/ParMultiDo(Anonymous)\n
 SqlTransform(beam:external:java:sql:v1)/BeamIOSourceRel_2/Convert.ConvertTransform/ParDo(Anonymous)/ParMultiDo(Anonymous):beam:transform:pardo:v1\n
 must follow: \n  downstream_side_inputs: <unknown>', 'external_4SqlTransform(beam:external:java:sql:v1)/BeamCalcRel_18/ParDo(Calc)/ParMultiDo(Calc)\n
 SqlTransform(beam:external:java:sql:v1)/BeamCalcRel_18/ParDo(Calc)/ParMultiDo(Calc):beam:transform:pardo:v1\n
 must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/Impulse_10\n
 assert_that/Create/Impulse:beam:transform:impulse:v1\n  must follow: \n  downstream_side_inputs:
<unknown>', 'ref_AppliedPTransform_assert_that/Create/FlatMap(<lambda at core.py:2623>)_11\n
 assert_that/Create/FlatMap(<lambda at core.py:2623>):beam:transform:pardo:v1\n  must
follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/Map(decode)_13\n
 assert_that/Create/Map(decode):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs:
<unknown>', 'ref_AppliedPTransform_assert_that/WindowInto(WindowIntoFn)_14\n  assert_that/WindowInto(WindowIntoFn):beam:transform:window_into:v1\n
 must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/ToVoidKey_15\n
 assert_that/ToVoidKey:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs:
<unknown>', 'ref_AppliedPTransform_assert_that/Group/pair_with_0_17\n  assert_that/Group/pair_with_0:beam:transform:pardo:v1\n
 must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/pair_with_1_18\n
 assert_that/Group/pair_with_1:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs:
<unknown>', 'ref_AppliedPTransform_assert_that/Group/Flatten_19\n  assert_that/Group/Flatten:beam:transform:flatten:v1\n
 must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/GroupByKey_20\n
 assert_that/Group/GroupByKey:beam:transform:group_by_key:v1\n  must follow: \n  downstream_side_inputs:
<unknown>', 'ref_AppliedPTransform_assert_that/Group/Map(_merge_tagged_vals_under_key)_21\n
 assert_that/Group/Map(_merge_tagged_vals_under_key):beam:transform:pardo:v1\n  must follow:
\n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Unkey_22\n
 assert_that/Unkey:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>',
'ref_AppliedPTransform_assert_that/Match_23\n  assert_that/Match:beam:transform:pardo:v1\n
 must follow: \n  downstream_side_inputs: <unknown>']
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_name' was already
added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'runner' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'temp_location' was already
added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'environment_cache_millis'
was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_endpoint' was already
added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'output_executable_path'
was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'sdk_worker_parallelism'
was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'dataflow_kms_key' was
already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'enable_streaming_engine'
was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'project' was already
added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_region' was already
added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_zone' was already
added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'files_to_stage' was already
added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'streaming' was already
added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'experiments' was already
added
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STOPPED
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STARTING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to RUNNING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
ok
test_row (apache_beam.transforms.sql_test.SqlTransformTest) ... INFO:apache_beam.utils.subprocess_server:Using
pre-built snapshot at <https://builds.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/sdks/java/extensions/sql/expansion-service/build/libs/beam-sdks-java-extensions-sql-expansion-service-2.23.0-SNAPSHOT.jar>
ERROR
test_tagged_join (apache_beam.transforms.sql_test.SqlTransformTest) ... INFO:apache_beam.utils.subprocess_server:Using
pre-built snapshot at <https://builds.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/sdks/java/extensions/sql/expansion-service/build/libs/beam-sdks-java-extensions-sql-expansion-service-2.23.0-SNAPSHOT.jar>
INFO:apache_beam.utils.subprocess_server:Starting service with ['java' '-jar' '<https://builds.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/sdks/java/extensions/sql/expansion-service/build/libs/beam-sdks-java-extensions-sql-expansion-service-2.23.0-SNAPSHOT.jar'>
'60337']
DEBUG:root:Waiting for grpc channel to be ready at localhost:60337.
INFO:apache_beam.utils.subprocess_server:Starting expansion service at localhost:60337
DEBUG:root:Waiting for grpc channel to be ready at localhost:60337.
INFO:apache_beam.utils.subprocess_server:	beam:external:java:sql:v1: org.apache.beam.sdk.expansion.service.ExpansionService$ExternalTransformRegistrarLoader$$Lambda$1/1327763628@593634ad
INFO:apache_beam.utils.subprocess_server:	beam:external:java:generate_sequence:v1: org.apache.beam.sdk.expansion.service.ExpansionService$ExternalTransformRegistrarLoader$$Lambda$1/1327763628@20fa23c1
DEBUG:root:Waiting for grpc channel to be ready at localhost:60337.
DEBUG:root:Waiting for grpc channel to be ready at localhost:60337.
DEBUG:root:Waiting for grpc channel to be ready at localhost:60337.
DEBUG:root:Waiting for grpc channel to be ready at localhost:60337.
INFO:apache_beam.utils.subprocess_server:Jun 10, 2020 6:31:08 PM org.apache.beam.sdk.expansion.service.ExpansionService
expand
INFO:apache_beam.utils.subprocess_server:INFO: Expanding 'SqlTransform(beam:external:java:sql:v1)'
with URN 'beam:external:java:sql:v1'
INFO:apache_beam.utils.subprocess_server:Jun 10, 2020 6:31:11 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner
convertToBeamRel
INFO:apache_beam.utils.subprocess_server:INFO: SQL:
INFO:apache_beam.utils.subprocess_server:SELECT `simple`.`id` AS `id`, `enrich`.`metadata`
AS `metadata`
INFO:apache_beam.utils.subprocess_server:FROM `beam`.`simple` AS `simple`
INFO:apache_beam.utils.subprocess_server:INNER JOIN `beam`.`enrich` AS `enrich` ON `simple`.`id`
= `enrich`.`id`
INFO:apache_beam.utils.subprocess_server:Jun 10, 2020 6:31:11 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner
convertToBeamRel
INFO:apache_beam.utils.subprocess_server:INFO: SQLPlan>
INFO:apache_beam.utils.subprocess_server:LogicalProject(id=[$0], metadata=[$4])
INFO:apache_beam.utils.subprocess_server:  LogicalJoin(condition=[=($0, $3)], joinType=[inner])
INFO:apache_beam.utils.subprocess_server:    BeamIOSourceRel(table=[[beam, simple]])
INFO:apache_beam.utils.subprocess_server:    BeamIOSourceRel(table=[[beam, enrich]])
INFO:apache_beam.utils.subprocess_server:
INFO:apache_beam.utils.subprocess_server:Jun 10, 2020 6:31:11 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner
convertToBeamRel
INFO:apache_beam.utils.subprocess_server:INFO: BEAMPlan>
INFO:apache_beam.utils.subprocess_server:BeamCalcRel(expr#0..4=[{inputs}], id=[$t2], metadata=[$t1])
INFO:apache_beam.utils.subprocess_server:  BeamCoGBKJoinRel(condition=[=($2, $0)], joinType=[inner])
INFO:apache_beam.utils.subprocess_server:    BeamIOSourceRel(table=[[beam, enrich]])
INFO:apache_beam.utils.subprocess_server:    BeamIOSourceRel(table=[[beam, simple]])
INFO:apache_beam.utils.subprocess_server:
DEBUG:root:Sending SIGINT to job_server
WARNING:root:Make sure that locally built Python SDK docker image has Python 2.7 interpreter.
INFO:root:Using Python SDK docker image: apache/beam_python2.7_sdk:2.23.0.dev. If the image
is not available at local, we will try to pull from hub.docker.com
WARNING:root:Make sure that locally built Python SDK docker image has Python 2.7 interpreter.
INFO:root:Using Python SDK docker image: apache/beam_python2.7_sdk:2.23.0.dev. If the image
is not available at local, we will try to pull from hub.docker.com
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function
lift_combiners at 0x7f91e7655488> ====================
DEBUG:apache_beam.runners.portability.fn_api_runner.translations:42 [1, 1, 1, 1, 1, 1, 1,
1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1,
1, 1, 1, 1, 1]
DEBUG:apache_beam.runners.portability.fn_api_runner.translations:Stages: ['ref_AppliedPTransform_Create
enrich/Impulse_3\n  Create enrich/Impulse:beam:transform:impulse:v1\n  must follow: \n  downstream_side_inputs:
<unknown>', 'ref_AppliedPTransform_Create enrich/FlatMap(<lambda at core.py:2623>)_4\n
 Create enrich/FlatMap(<lambda at core.py:2623>):beam:transform:pardo:v1\n  must follow:
\n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create enrich/MaybeReshuffle/Reshuffle/AddRandomKeys_7\n
 Create enrich/MaybeReshuffle/Reshuffle/AddRandomKeys:beam:transform:pardo:v1\n  must follow:
\n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create enrich/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps)_9\n
 Create enrich/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps):beam:transform:pardo:v1\n
 must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create
enrich/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey_10\n  Create enrich/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey:beam:transform:group_by_key:v1\n
 must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create
enrich/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps)_11\n  Create enrich/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps):beam:transform:pardo:v1\n
 must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create
enrich/MaybeReshuffle/Reshuffle/RemoveRandomKeys_12\n  Create enrich/MaybeReshuffle/Reshuffle/RemoveRandomKeys:beam:transform:pardo:v1\n
 must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create
enrich/Map(decode)_13\n  Create enrich/Map(decode):beam:transform:pardo:v1\n  must follow:
\n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create simple/Impulse_15\n
 Create simple/Impulse:beam:transform:impulse:v1\n  must follow: \n  downstream_side_inputs:
<unknown>', 'ref_AppliedPTransform_Create simple/FlatMap(<lambda at core.py:2623>)_16\n
 Create simple/FlatMap(<lambda at core.py:2623>):beam:transform:pardo:v1\n  must follow:
\n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create simple/MaybeReshuffle/Reshuffle/AddRandomKeys_19\n
 Create simple/MaybeReshuffle/Reshuffle/AddRandomKeys:beam:transform:pardo:v1\n  must follow:
\n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create simple/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps)_21\n
 Create simple/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps):beam:transform:pardo:v1\n
 must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create
simple/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey_22\n  Create simple/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey:beam:transform:group_by_key:v1\n
 must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create
simple/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps)_23\n  Create simple/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps):beam:transform:pardo:v1\n
 must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create
simple/MaybeReshuffle/Reshuffle/RemoveRandomKeys_24\n  Create simple/MaybeReshuffle/Reshuffle/RemoveRandomKeys:beam:transform:pardo:v1\n
 must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create
simple/Map(decode)_25\n  Create simple/Map(decode):beam:transform:pardo:v1\n  must follow:
\n  downstream_side_inputs: <unknown>', 'external_6SqlTransform(beam:external:java:sql:v1)/BeamIOSourceRel_6/Convert.ConvertTransform/ParDo(Anonymous)/ParMultiDo(Anonymous)\n
 SqlTransform(beam:external:java:sql:v1)/BeamIOSourceRel_6/Convert.ConvertTransform/ParDo(Anonymous)/ParMultiDo(Anonymous):beam:transform:pardo:v1\n
 must follow: \n  downstream_side_inputs: <unknown>', 'external_6SqlTransform(beam:external:java:sql:v1)/BeamIOSourceRel_5/Convert.ConvertTransform/ParDo(Anonymous)/ParMultiDo(Anonymous)\n
 SqlTransform(beam:external:java:sql:v1)/BeamIOSourceRel_5/Convert.ConvertTransform/ParDo(Anonymous)/ParMultiDo(Anonymous):beam:transform:pardo:v1\n
 must follow: \n  downstream_side_inputs: <unknown>', 'external_6SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/left_TimestampCombiner/Flatten.PCollections\n
 SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/left_TimestampCombiner/Flatten.PCollections:beam:transform:flatten:v1\n
 must follow: \n  downstream_side_inputs: <unknown>', 'external_6SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/right_TimestampCombiner/Flatten.PCollections\n
 SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/right_TimestampCombiner/Flatten.PCollections:beam:transform:flatten:v1\n
 must follow: \n  downstream_side_inputs: <unknown>', 'external_6SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/Join.Impl/CoGroup.ExpandCrossProduct/extractKeylhs/ParMultiDo(Anonymous)\n
 SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/Join.Impl/CoGroup.ExpandCrossProduct/extractKeylhs/ParMultiDo(Anonymous):beam:transform:pardo:v1\n
 must follow: \n  downstream_side_inputs: <unknown>', 'external_6SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/Join.Impl/CoGroup.ExpandCrossProduct/extractKeyrhs/ParMultiDo(Anonymous)\n
 SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/Join.Impl/CoGroup.ExpandCrossProduct/extractKeyrhs/ParMultiDo(Anonymous):beam:transform:pardo:v1\n
 must follow: \n  downstream_side_inputs: <unknown>', 'external_6SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/Join.Impl/CoGroup.ExpandCrossProduct/CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable)\n
 SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/Join.Impl/CoGroup.ExpandCrossProduct/CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable):beam:transform:pardo:v1\n
 must follow: \n  downstream_side_inputs: <unknown>', 'external_6SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/Join.Impl/CoGroup.ExpandCrossProduct/CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable)\n
 SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/Join.Impl/CoGroup.ExpandCrossProduct/CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable):beam:transform:pardo:v1\n
 must follow: \n  downstream_side_inputs: <unknown>', 'external_6SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/Join.Impl/CoGroup.ExpandCrossProduct/CoGroupByKey/Flatten\n
 SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/Join.Impl/CoGroup.ExpandCrossProduct/CoGroupByKey/Flatten:beam:transform:flatten:v1\n
 must follow: \n  downstream_side_inputs: <unknown>', 'external_6SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/Join.Impl/CoGroup.ExpandCrossProduct/CoGroupByKey/GBK\n
 SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/Join.Impl/CoGroup.ExpandCrossProduct/CoGroupByKey/GBK:beam:transform:group_by_key:v1\n
 must follow: \n  downstream_side_inputs: <unknown>', 'external_6SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/Join.Impl/CoGroup.ExpandCrossProduct/CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult)\n
 SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/Join.Impl/CoGroup.ExpandCrossProduct/CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult):beam:transform:pardo:v1\n
 must follow: \n  downstream_side_inputs: <unknown>', 'external_6SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/Join.Impl/CoGroup.ExpandCrossProduct/ParDo(ConvertCoGbkResult)/ParMultiDo(ConvertCoGbkResult)\n
 SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/Join.Impl/CoGroup.ExpandCrossProduct/ParDo(ConvertCoGbkResult)/ParMultiDo(ConvertCoGbkResult):beam:transform:pardo:v1\n
 must follow: \n  downstream_side_inputs: <unknown>', 'external_6SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/Select.Fields/ParDo(Select)/ParMultiDo(Select)\n
 SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/Select.Fields/ParDo(Select)/ParMultiDo(Select):beam:transform:pardo:v1\n
 must follow: \n  downstream_side_inputs: <unknown>', 'external_6SqlTransform(beam:external:java:sql:v1)/BeamCalcRel_96/ParDo(Calc)/ParMultiDo(Calc)\n
 SqlTransform(beam:external:java:sql:v1)/BeamCalcRel_96/ParDo(Calc)/ParMultiDo(Calc):beam:transform:pardo:v1\n
 must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/Impulse_29\n
 assert_that/Create/Impulse:beam:transform:impulse:v1\n  must follow: \n  downstream_side_inputs:
<unknown>', 'ref_AppliedPTransform_assert_that/Create/FlatMap(<lambda at core.py:2623>)_30\n
 assert_that/Create/FlatMap(<lambda at core.py:2623>):beam:transform:pardo:v1\n  must
follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/Map(decode)_32\n
 assert_that/Create/Map(decode):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs:
<unknown>', 'ref_AppliedPTransform_assert_that/WindowInto(WindowIntoFn)_33\n  assert_that/WindowInto(WindowIntoFn):beam:transform:window_into:v1\n
 must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/ToVoidKey_34\n
 assert_that/ToVoidKey:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs:
<unknown>', 'ref_AppliedPTransform_assert_that/Group/pair_with_0_36\n  assert_that/Group/pair_with_0:beam:transform:pardo:v1\n
 must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/pair_with_1_37\n
 assert_that/Group/pair_with_1:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs:
<unknown>', 'ref_AppliedPTransform_assert_that/Group/Flatten_38\n  assert_that/Group/Flatten:beam:transform:flatten:v1\n
 must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/GroupByKey_39\n
 assert_that/Group/GroupByKey:beam:transform:group_by_key:v1\n  must follow: \n  downstream_side_inputs:
<unknown>', 'ref_AppliedPTransform_assert_that/Group/Map(_merge_tagged_vals_under_key)_40\n
 assert_that/Group/Map(_merge_tagged_vals_under_key):beam:transform:pardo:v1\n  must follow:
\n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Unkey_41\n
 assert_that/Unkey:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>',
'ref_AppliedPTransform_assert_that/Match_42\n  assert_that/Match:beam:transform:pardo:v1\n
 must follow: \n  downstream_side_inputs: <unknown>']
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_name' was already
added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'runner' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'temp_location' was already
added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'environment_cache_millis'
was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_endpoint' was already
added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'output_executable_path'
was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'sdk_worker_parallelism'
was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'dataflow_kms_key' was
already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'enable_streaming_engine'
was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'project' was already
added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_region' was already
added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_zone' was already
added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'files_to_stage' was already
added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'streaming' was already
added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'experiments' was already
added
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STOPPED
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STARTING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to RUNNING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
ok
test_zetasql_generate_data (apache_beam.transforms.sql_test.SqlTransformTest) ... INFO:apache_beam.utils.subprocess_server:Using
pre-built snapshot at <https://builds.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/sdks/java/extensions/sql/expansion-service/build/libs/beam-sdks-java-extensions-sql-expansion-service-2.23.0-SNAPSHOT.jar>
INFO:apache_beam.utils.subprocess_server:Starting service with ['java' '-jar' '<https://builds.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/sdks/java/extensions/sql/expansion-service/build/libs/beam-sdks-java-extensions-sql-expansion-service-2.23.0-SNAPSHOT.jar'>
'56595']
DEBUG:root:Waiting for grpc channel to be ready at localhost:56595.
INFO:apache_beam.utils.subprocess_server:Starting expansion service at localhost:56595
INFO:apache_beam.utils.subprocess_server:	beam:external:java:sql:v1: org.apache.beam.sdk.expansion.service.ExpansionService$ExternalTransformRegistrarLoader$$Lambda$1/1327763628@593634ad
INFO:apache_beam.utils.subprocess_server:	beam:external:java:generate_sequence:v1: org.apache.beam.sdk.expansion.service.ExpansionService$ExternalTransformRegistrarLoader$$Lambda$1/1327763628@20fa23c1
DEBUG:root:Waiting for grpc channel to be ready at localhost:56595.
DEBUG:root:Waiting for grpc channel to be ready at localhost:56595.
DEBUG:root:Waiting for grpc channel to be ready at localhost:56595.
DEBUG:root:Waiting for grpc channel to be ready at localhost:56595.
DEBUG:root:Waiting for grpc channel to be ready at localhost:56595.
INFO:apache_beam.utils.subprocess_server:Jun 10, 2020 6:31:32 PM org.apache.beam.sdk.expansion.service.ExpansionService
expand
INFO:apache_beam.utils.subprocess_server:INFO: Expanding 'SqlTransform(beam:external:java:sql:v1)'
with URN 'beam:external:java:sql:v1'
DEBUG:root:Sending SIGINT to job_server
WARNING:root:Make sure that locally built Python SDK docker image has Python 2.7 interpreter.
INFO:root:Using Python SDK docker image: apache/beam_python2.7_sdk:2.23.0.dev. If the image
is not available at local, we will try to pull from hub.docker.com
WARNING:root:Make sure that locally built Python SDK docker image has Python 2.7 interpreter.
INFO:root:Using Python SDK docker image: apache/beam_python2.7_sdk:2.23.0.dev. If the image
is not available at local, we will try to pull from hub.docker.com
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function
lift_combiners at 0x7f91e7655488> ====================
DEBUG:apache_beam.runners.portability.fn_api_runner.translations:16 [1, 1, 1, 1, 1, 1, 1,
1, 1, 1, 1, 1, 1, 1, 1, 1]
DEBUG:apache_beam.runners.portability.fn_api_runner.translations:Stages: ['external_7SqlTransform(beam:external:java:sql:v1)/BeamValuesRel_13/Create.Values/Read(CreateSource)/Impulse\n
 SqlTransform(beam:external:java:sql:v1)/BeamValuesRel_13/Create.Values/Read(CreateSource)/Impulse:beam:transform:impulse:v1\n
 must follow: \n  downstream_side_inputs: <unknown>', 'external_7SqlTransform(beam:external:java:sql:v1)/BeamValuesRel_13/Create.Values/Read(CreateSource)/MapElements/Map/ParMultiDo(Anonymous)\n
 SqlTransform(beam:external:java:sql:v1)/BeamValuesRel_13/Create.Values/Read(CreateSource)/MapElements/Map/ParMultiDo(Anonymous):beam:transform:pardo:v1\n
 must follow: \n  downstream_side_inputs: <unknown>', 'external_7SqlTransform(beam:external:java:sql:v1)/BeamValuesRel_13/Create.Values/Read(CreateSource)/ParDo(BoundedSourceAsSDFWrapper)/ParMultiDo(BoundedSourceAsSDFWrapper)\n
 SqlTransform(beam:external:java:sql:v1)/BeamValuesRel_13/Create.Values/Read(CreateSource)/ParDo(BoundedSourceAsSDFWrapper)/ParMultiDo(BoundedSourceAsSDFWrapper):beam:transform:pardo:v1\n
 must follow: \n  downstream_side_inputs: <unknown>', 'external_7SqlTransform(beam:external:java:sql:v1)/BeamZetaSqlCalcRel_17/ParDo(Calc)/ParMultiDo(Calc)\n
 SqlTransform(beam:external:java:sql:v1)/BeamZetaSqlCalcRel_17/ParDo(Calc)/ParMultiDo(Calc):beam:transform:pardo:v1\n
 must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/Impulse_5\n
 assert_that/Create/Impulse:beam:transform:impulse:v1\n  must follow: \n  downstream_side_inputs:
<unknown>', 'ref_AppliedPTransform_assert_that/Create/FlatMap(<lambda at core.py:2623>)_6\n
 assert_that/Create/FlatMap(<lambda at core.py:2623>):beam:transform:pardo:v1\n  must
follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/Map(decode)_8\n
 assert_that/Create/Map(decode):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs:
<unknown>', 'ref_AppliedPTransform_assert_that/WindowInto(WindowIntoFn)_9\n  assert_that/WindowInto(WindowIntoFn):beam:transform:window_into:v1\n
 must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/ToVoidKey_10\n
 assert_that/ToVoidKey:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs:
<unknown>', 'ref_AppliedPTransform_assert_that/Group/pair_with_0_12\n  assert_that/Group/pair_with_0:beam:transform:pardo:v1\n
 must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/pair_with_1_13\n
 assert_that/Group/pair_with_1:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs:
<unknown>', 'ref_AppliedPTransform_assert_that/Group/Flatten_14\n  assert_that/Group/Flatten:beam:transform:flatten:v1\n
 must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/GroupByKey_15\n
 assert_that/Group/GroupByKey:beam:transform:group_by_key:v1\n  must follow: \n  downstream_side_inputs:
<unknown>', 'ref_AppliedPTransform_assert_that/Group/Map(_merge_tagged_vals_under_key)_16\n
 assert_that/Group/Map(_merge_tagged_vals_under_key):beam:transform:pardo:v1\n  must follow:
\n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Unkey_17\n
 assert_that/Unkey:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>',
'ref_AppliedPTransform_assert_that/Match_18\n  assert_that/Match:beam:transform:pardo:v1\n
 must follow: \n  downstream_side_inputs: <unknown>']
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_name' was already
added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'runner' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'temp_location' was already
added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'environment_cache_millis'
was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_endpoint' was already
added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'output_executable_path'
was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'sdk_worker_parallelism'
was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'dataflow_kms_key' was
already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'enable_streaming_engine'
was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'project' was already
added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_region' was already
added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_zone' was already
added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'files_to_stage' was already
added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'streaming' was already
added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'experiments' was already
added
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STOPPED
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STARTING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to RUNNING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
ok

======================================================================
ERROR: test_row (apache_beam.transforms.sql_test.SqlTransformTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "<https://builds.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/sdks/python/apache_beam/transforms/sql_test.py",>
line 147, in test_row
    | SqlTransform("SELECT a*a as s, LENGTH(b) AS c FROM PCOLLECTION"))
  File "<https://builds.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/sdks/python/apache_beam/pvalue.py",>
line 141, in __or__
    return self.pipeline.apply(ptransform, self)
  File "<https://builds.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/sdks/python/apache_beam/pipeline.py",>
line 655, in apply
    pvalueish_result = self.runner.apply(transform, pvalueish, self._options)
  File "<https://builds.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/sdks/python/apache_beam/runners/runner.py",>
line 198, in apply
    return m(transform, input, options)
  File "<https://builds.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/sdks/python/apache_beam/runners/runner.py",>
line 228, in apply_PTransform
    return transform.expand(input)
  File "<https://builds.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/sdks/python/apache_beam/transforms/external.py",>
line 295, in expand
    transform_proto.inputs[tag] = context.pcollections.get_id(pcoll)
  File "<https://builds.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/sdks/python/apache_beam/runners/pipeline_context.py",>
line 92, in get_id
    self._id_to_proto[id] = obj.to_runner_api(self._pipeline_context)
  File "<https://builds.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/sdks/python/apache_beam/pvalue.py",>
line 190, in to_runner_api
    coder_id=context.coder_id_from_element_type(self.element_type),
  File "<https://builds.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/sdks/python/apache_beam/runners/pipeline_context.py",>
line 219, in coder_id_from_element_type
    return self.coders.get_id(coders.registry.get_coder(element_type))
  File "<https://builds.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/sdks/python/apache_beam/coders/typecoders.py",>
line 130, in get_coder
    return coder.from_type_hint(typehint, self)
  File "<https://builds.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/sdks/python/apache_beam/coders/row_coder.py",>
line 90, in from_type_hint
    schema = named_fields_to_schema(type_hint._fields)
  File "<https://builds.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/sdks/python/apache_beam/typehints/schemas.py",>
line 127, in named_fields_to_schema
    for (name, type) in names_and_types
  File "<https://builds.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/sdks/python/apache_beam/typehints/schemas.py",>
line 157, in typing_to_runner_api
    "type 'str' is not supported in python 2. Please use 'unicode' or "
ValueError: type 'str' is not supported in python 2. Please use 'unicode' or 'typing.ByteString'
instead to unambiguously indicate if this is a UTF-8 string or a byte array.
-------------------- >> begin captured logging << --------------------
apache_beam.utils.subprocess_server: INFO: Using pre-built snapshot at <https://builds.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/sdks/java/extensions/sql/expansion-service/build/libs/beam-sdks-java-extensions-sql-expansion-service-2.23.0-SNAPSHOT.jar>
--------------------- >> end captured logging << ---------------------

----------------------------------------------------------------------
XML: nosetests-xlangSqlValidateRunner.xml
----------------------------------------------------------------------
XML: <https://builds.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/sdks/python/nosetests.xml>
----------------------------------------------------------------------
Ran 7 tests in 138.364s

FAILED (errors=1)

> Task :runners:spark:job-server:validatesCrossLanguageRunnerPythonUsingSql FAILED

> Task :runners:spark:job-server:sparkJobServerCleanup
Stopping job server pid: 27028.

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':runners:spark:job-server:validatesCrossLanguageRunnerPythonUsingSql'.
> Process 'command 'sh'' finished with non-zero exit value 1

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to
get more log output. Run with --scan to get full insights.

* Get more help at https://help.gradle.org

Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings

BUILD FAILED in 25m 56s
134 actionable tasks: 104 executed, 28 from cache, 2 up-to-date

Publishing build scan...
https://gradle.com/s/ltmt5hplxpedm

Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure

---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org


Mime
View raw message