beam-builds mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Apache Jenkins Server <>
Subject Build failed in Jenkins: beam_PostCommit_Java_ValidatesRunner_Twister2 #8
Date Tue, 19 May 2020 18:03:53 GMT
See <>


[amogh.tiwari] lzo-addons

[markliu] [BEAM-8831] Do not copy generated files for Python source copy

[amogh.tiwari] 3rd dec 2019, 12:43AM

[amogh.tiwari] PR corrections

[amogh.tiwari] PR javaPreCommit update

[amogh.tiwari] PR changes: added testLzopSpilttale()

[git] Remove optionality and add sensible defaults to PubsubIO builders.

[amogh.tiwari] updated gradle for supporting optional dependency of lzo- 2:39 AM IST

[jkai] [BEAM-8331] rewrite the calcite JDBC urls

[iemejia] [BEAM-9162] Upgrade Jackson to version 2.10.2

[boyuanz] Update verify_release_build script to run python tests with dev version.

[robertwb] Supporting infrastructure for dataframes on beam.

[robertwb] Basic deferred data frame implementation.

[robertwb] yapf, py2

[millsd] Update Dataflow monitoring URL

[filiperegadas] Add BigQuery useAvroLogicalTypes option

[filiperegadas] fixup! Add BigQuery useAvroLogicalTypes option

[amogh.tiwari] update 19/02/2020 2:32 AM added static class, removed wrappers, updated

[heejong] [BEAM-9335] update hard-coded coder id when translating Java external

[iemejia] Mark Test categories as internal and improve categorization

[robertwb] Update go beam runner generated protos.

[github] Update

[github] Update

[github] Update

[github] Update

[github] Update

[github] Update

[github] Update

[github] Update

[shubham.srivastava] finishing touch 20/02/2020 6:43PM

[robertwb] Add standard protocol capabilities to protos.

[kcweaver] [BEAM-9225] Fix Flink uberjar job termination bug.

[kcweaver] Reuse get_state method.

[robertwb] [BEAM-9339] Declare capabilities for Go SDK.

[lcwik] [BEAM-5605] Eagerly close the BoundedReader once we have read everything

[lcwik] fixup! Fix SpotBugs failure

[kcweaver] [BEAM-9022] publish Spark job server Docker image

[drubinstein] Bump google cloud bigquery to 1.24.0

[github] Revert "[BEAM-9085] Fix performance regression in SyntheticSource

[github] [BEAM-8537] Provide WatermarkEstimator to track watermark (#10375)

[github] Make sure calling try_claim(0) more than once also trows exception.

[robertwb] [BEAM-9339] Declare capabilities for Python SDK.

[robertwb] Add some standard requirement URNs to the protos.

[kcweaver] [BEAM-9356] reduce Flink test logs to warn

[github] [BEAM-9063] migrate docker images to apache (#10612)

[github] [BEAM-9252] Exclude jboss's Main and (#10930)

[boyuanz] Clean up and add type-hints to SDF API

[robertwb] [BEAM-9340] Populate requirements for Python DoFn properties.

[hannahjiang] fix postcommit failure

[robertwb] [BEAM-8019] Branch on having multiple environments.

[github] [BEAM-9359] Switch to Data Catalog client (#10917)

[github] [BEAM-9344] Add support for bundle finalization execution to the Beam

[jvilcek] [BEAM-9360] Fix equivalence check for FieldType

[iemejia] [BEAM-9342] Upgrade vendored bytebuddy to version 1.10.8

[chadrik] Create a class to encapsulate the work required to submit a pipeline to

[iemejia] Add Dataflow Java11 ValidatesRunner badge to the PR template

[github] typings and docs for

[github] Merge pull request #10944: [BEAM-7274] optimize oneOf handling

[github] [BEAM-8280] Fix IOTypeHints origin traceback on partials (#10927)

[relax] Support null fields in rows with ByteBuddy generated code.

[robertwb] Allow metrics update to be tolerant to uninitalized metric containers.

[github] [GoSDK] Fix race condition in statemgr & test (#10941)

[rohde.samuel] Move TestStream implementation to replacement transform

[github] [BEAM-9347] Don't overwrite default runner harness for unified worker

[boyuanz] Update docstring of ManualWatermarkEstimator.set_watermark()

[kcweaver] [BEAM-9373] Spark/Flink tests fix string concat

[boyuanz] Address comments

[boyuanz] Address comments again

[github] [BEAM-9228] Support further partition for FnApi ListBuffer (#10847)

[github] [BEAM-7926] Data-centric Interactive Part3 (#10731)

[boyuanz] Use NoOpWatermarkEstimator in sdf_direct_runner

[chamikara] Updates Dataflow client

[github] [BEAM-9240]: Check for Nullability in typesEqual() method of FieldType

[amogh.tiwari] 25/02/2020 updated imports Amogh Tiwari & Shubham Srivastava

[iemejia] [BEAM-8616] Make hadoop-client a provided dependency on ParquetIO

[mxm] [BEAM-9345] Remove workaround to restore stdout/stderr during JobGraph

[iemejia] [BEAM-9364] Refactor KafkaIO to use DeserializerProviders

[mxm] [BEAM-9345] Add end-to-end Flink job submission test

[iemejia] [BEAM-9352] Align version of transitive jackson dependencies with Beam

[michal.walenia] [BEAM-9258] Add integration test for Cloud DLP

[iemejia] [BEAM-9329] Support request of schemas by version on KafkaIO + CSR

[lcwik] [BEAM-9252] Update to vendored gRPC without problematic

[github] Update

[github] Update

[github] Update

[lcwik] [BEAM-2822, BEAM-2939, BEAM-6189, BEAM-4374] Enable passing completed

[crites] Changes TestStreamTranscriptTest to only emit two elements so that its

[chamikara] Logs BQ insert failures

[alex] [BEAM-7274] Add DynamicMessage Schema support

[github] [BEAM-9322] Fix tag output names within Dataflow to be consistent with

[iemejia] [BEAM-9342] Exclude module-info.class from vendored Byte Buddy 1.10.8

[iemejia] Add KafkaIO support for Confluent Schema Registry to the CHANGEs file

[github] [BEAM-9247] Integrate GCP Vision API functionality (#10959)

[github] Fix kotlin warnings (#10976)

[github] Update python sdk container version to beam-master-20200225 (#10965)

[github] [BEAM-9248] Integrate Google Cloud Natural Language functionality for

[iemejia] Refine access level for `sdks/java/extensions/protobuf`

[github] [BEAM-9355] Basic support for NewType (#10928)

[github] [BEAM-8979] reintroduce mypy-protobuf stub generation (#10734)

[github] [BEAM-8335] Background Caching job (#10899)

[crites] Clean up of TestStreamTranscriptTests. Remvoes check for final field in

[github] [BEAM-8458] Add option to set temp dataset in BigQueryIO.Read (#9852)

[iemejia] Make logger naming consistent with Apache Beam LOG standard

[kcweaver] [BEAM-9300] convert struct literal in ZetaSQL

[github] fix breakage (#10934)

[github] Merge pull request #10901 from [BEAM-8965] Remove duplicate sideinputs

[pabloem] Fix formatting

[github] [BEAM-8618] Tear down unused DoFns periodically in Python SDK harness.

[alex] [BEAM-9394] DynamicMessage handling of empty map violates schema

[github] Merge pull request #10854: State timers documentation

[iemejia] [BEAM-9384] Add SchemaRegistry.getSchemaCoder to get SchemaCoders for

[lcwik] [BEAM-5524] Fix minor issue in style guide.

[lcwik] [BEAM-9397] Pass all but output receiver parameters to start

[github] [BEAM-8201] Pass all other endpoints through provisioning service.

[suztomo] Linkage Checker 1.1.4

[kcweaver] [BEAM-9401] bind Flink MiniCluster to localhost

[robinyqiu] Bump Dataflow Java worker container version

[kcweaver] Test schema does not need to be nullable.

[sunjincheng121] [BEAM-9288] Not bundle conscrypt in gRPC vendor

[mxm] [BEAM-9345] Fix source of test flakiness in FlinkSubmissionTest

[github] [BEAM-9396] Match Docker image names in Jenkins jobs with those

[github] [BEAM-9392] Fix Multi TestStream assertion errors (#10982)

[kamil.wasilewski] Add integration test for AnnotateImage transform

[github] Add integration test for AnnotateText transform (#10977)

[chadrik] [BEAM-9405] Fix post-commit error about create_job_service

[chadrik] more typing fixes

[chadrik] Fix typing issue with python 3.5.2

[chadrik] fixes

[chadrik] Address more issues discovered after rebase

[chadrik] Improve the idiom used for conditional imports

[chadrik] Fix more issues

[chadrik] Update to latest mypy version

[amaliujia] Moving to 2.21.0-SNAPSHOT on master branch.

[github] [BEAM-8487] Handle nested forward references (#10932)

[github] [BEAM-9287] Add Postcommit tests for dataflow runner v2  (#10998)

[chadrik] [BEAM-7746] Runtime change to timestamp/duration equality

[github] Adds DisplayData for StateSpecs used by stateful ParDos

[iemejia] Fix non correctly formatted class in sdks/java/core

[iemejia] [BEAM-9342[ Update bytebuddy to version 1.10.8

[] [BEAM-8925] Tika version update to 1.23

[12602502+Ardagan] [BEAM-8327] Override Gradle cache for community metrics prober

[ehudm] Reduce warnings in pytest runs.

[heejong] [BEAM-9415] fix postcommit xvr tests

[github] Merge pull request #10968 from [BEAM-9381] Adding display data to

[github] [BEAM-8335] Add PCollection to DataFrame logic for InteractiveRunner.

[robertwb] Remove excessive logging.

[github] [BEAM-2939] Java UnboundedSource SDF wrapper (#10897)

[iemejia] [website] Update link to environment_type (SDK harness configuration)

[iemejia] Fix typo on python code

[kamil.wasilewski] Fix: skip test if GCP dependencies are not installed

[fernandodiaz] [BEAM-9424] Allow grouping by LogicalType

[github] Revert "[BEAM-8335] Add PCollection to DataFrame logic for

[echauchot] Add metrics export to documentation on the website.

[github] [BEAM-8382] Add rate limit policy to KinesisIO.Read (#9765)

[lcwik] [BEAM-9288] Bump version number vendored gRPC build.

[chadrik] [BEAM-9274] Support running yapf in a git pre-commit hook

[rohde.samuel] [BEAM-8335] Add PCollection to Dataframe logic for InteractiveRunner.

[github] [BEAM-8575] Modified trigger test to work for different runners.

[github] [BEAM-9413] fix beam_PostCommit_Py_ValCon (#11023)

[rohde.samuel] ReverseTestStream Implementation

[github] Update lostluck's info on the Go SDK roadmap

[suztomo] Google-cloud-bigquery 1.108.0

[github] [BEAM-9432] Move expansion service into its own project. (#11035)

[ehudm] [BEAM-3713] Remove nosetests from tox.ini

[github] Merge pull request #11025: [BEAM-6428] Improve select performance with

[github] Switch contact email to

[github] [BEAM-6374] Emit PCollection metrics from GoSDK (#10942)

[amaliujia] [BEAM-9288] Not bundle conscrypt in gRPC vendor in META-INF/

[kcweaver] [BEAM-9448] Fix log message for job server cache.

[github] Update container image tags used by Dataflow runner for Beam master

[github] [BEAM-8328] Disable community metrics integration test in 'test' task

[iemejia] [BEAM-9450] Update links to

[iemejia] [BEAM-9450] Convert links available via https to use https

[github] Add integration test for AnnotateVideoWithContext transform (#10986)

[lcwik] [BEAM-9452] Update classgraph to latest version to resolve windows

[hktang] [BEAM-9453] Changed new string creation to use StandardCharsets.UTF_8

[chuck.yang] Use Avro format for file loads to BigQuery

[jkai] [Hotfix] fix rabbitmp spotless check

[kcweaver] Downgrade cache log level from warn->info.

[github] Revert "[BEAM-6374] Emit PCollection metrics from GoSDK (#10942)"

[github] Merge pull request #11032 from [BEAM-8335] Display rather than logging

[github] Fix a bug in performance test for reading data from BigQuery (#11062)

[suztomo] grpc 1.27.2 and gax 1.54.0

[suztomo] bigquerystorage 0.125.0-beta

[apilloud] [BEAM-9463] Bump ZetaSQL to 2020.03.1

[lcwik] [BEAM-2939, BEAM-9458] Add deduplication transform for SplittableDoFns

[lcwik] [BEAM-9464] Fix WithKeys to respect parameterized types

[ankurgoenka] [BEAM-9465] Fire repeatedly in reshuffle

[lcwik] [BEAM-2939, BEAM-9458] Use deduplication transform for UnboundedSources

[echauchot] Fix wrong generated code comment.

[github] [BEAM-9396] Fix Docker image name in CoGBK test for Python on Flink

[lcwik] [BEAM-9288] Update to use vendored gRPC without shaded conscrypt

[github] [BEAM-9319] Clean up start topic in TestPubsubSignal (#11072)

[lcwik] [BEAM-2939] Follow-up on comment in pr/11065

[lcwik] [BEAM-9473] Dont copy over META-INF index/checksum/signing files during

[apilloud] [BEAM-9411] Enable BigQuery DIRECT_READ by default in SQL

[hannahjiang] update for 2.20

[lcwik] [BEAM-9475] Fix typos and shore up expectations on type

[rohde.samuel] BEAM[8335] TestStreamService integration with DirectRunner

[github] [BEAM-7926] Update Data Visualization (#11020)

[ankurgoenka] [BEAM-9402] Remove options overwrite

[chadrik] Add pre-commit hook for pylint

[github] Additional new Python Katas (#11078)

[github] [BEAM-9478] Update samza runner page to reflect post 1.0 changes

[suztomo] grpc-google-cloud-pubsub-v1 1.85.1

[pabloem] Updating BigQuery client APIs

[github] [BEAM-9481] Exclude signature files from expansion service test

[github] Install typing package only for Python < 3.5.3 (#10821)

[heejong] [BEAM-9056] Staging artifacts from environment

[sunjincheng121] [BEAM-9295] Add Flink 1.10 build target and Make FlinkRunner compatible

[ankurgoenka] [BEAM-9485] Raise error when transform urn is not implemented

[12602502+Ardagan] [BEAM-9431] Remove ReadFromPubSub/Read-out0-ElementCount from the

[crites] Adds clearing of pane info state when windows get merged away.

[github] Update Python roadmap for 2.7 eol

[mxm] [BEAM-9474] Improve robustness of BundleFactory and ProcessEnvironment

[github] [BEAM-7815] update MemoryReporter comments about using guppy3 (#11073)

[rohde.samuel] [BEAM-8335] Modify the StreamingCache to subclass the CacheManager

[zyichi] Update BigQuery source in bigquery_tornadoes example

[sunjincheng121] [BEAM-9298] Drop support for Flink 1.7

[github] Fixing at head

[mxm] [BEAM-9490] Guard referencing for environment expiration via a lock

[github] Verify schema early in ToJson and JsonToRow (#11105)

[ehudm] [BEAM-9398] runtime_type_check: support setup

[lcwik] [BEAM-9481] fix indentation

[github] Merge pull request #11103 from [BEAM-9494] Reifying outputs from BQ file

[github] [BEAM-8335] Implemented Capture Size limitation (#11050)

[github] [BEAM-9294] Move RowJsonException out of RowJsonSerializer (#11102)

[github] Merge pull request #11046: [BEAM-9442] Properly handle nullable fields

[relax] switch cogbk to use Beam transform

[ankurgoenka] [BEAM-9287] disable validates runner test which uses teststreams for

[relax] finish join

[sunjincheng121] [BEAM-9299-PR]Upgrade Flink Runner 1.8x to 1.8.3 and 1.9x to 1.9.2

[lcwik] [BEAM-2939] Implement interfaces and concrete watermark estimators

[jfarr1] [BEAM-8374] Add alternate SnsIO PublishResult coders

[ankurgoenka] [BEAM-9499] Sickbay test_multi_triggered_gbk_side_input for streaming

[kcweaver] [BEAM-9503] Insert missing comma in process worker script.

[robertwb] Minor cleanup, lint.

[robertwb] [BEAM-9433] Create expansion service artifact for common Java IOs.

[thw] [BEAM-9490] Use the lock that belongs to the cache when bundle load

[github] Update Dataflow py container version (#11120)

[github] [BEAM-7923] Streaming support and pipeline pruning when instrumenting a

[github] Update default value in Java snippet

[ankurgoenka] [BEAM-9504] Sickbay streaming test for batch VR

[rohde.samuel] [BEAM-8335] Final PR to merge the InteractiveBeam feature branch

[github] [BEAM-9477] RowCoder should be hashable and picklable (#11088)

[apilloud] [BEAM-8057] Reject Infinite or NaN literals at parse time

[apilloud] [BEAM-7832] Translate ZetaSQL joins without condition

[robertwb] Log in a daemon thread.

[thw] [BEAM-8815] Skip removal of manifest when no artifacts were staged.

[robertwb] [BEAM-9496] Evaluation of deferred dataframes via Beam operations.

[relax] support side-input joins

[relax] support side-input joins

[relax] spotless

[github] [BEAM-9346] Improve the efficiency of TFRecordIO (#11122)

[kawaigin] [BEAM-8335] Refactor IPythonLogHandler

[kcweaver] [BEAM-8866] Use unique temp dir for output of portable word count tests.

[apilloud] [BEAM-8070] Preserve type for empty array

[kcweaver] [BEAM-9509] Improve error message for bad job server URL.

[relax] make FieldAccessDescriptor always be field-insertion order

[davidyan] [BEAM-9510] Fixing version incompatibities in

[github] Merge pull request #10991 [BEAM-3301] Refactor DoFn validation & allow

[davidyan] Bring the dep versions up to par with

[valentyn] Install typing only on 3.5.2 or earlier versions of Python.

[github] Update dataflow py container ver to 20200317 (#11145)

[kawaigin] [BEAM-7923] Include side effects in

[lcwik] [BEAM-9430] Migrate from ProcessContext#updateWatermark to

[github] [BEAM-9498] Include descriptor and type of unsupported fields in RowJson

[github] Merge pull request #11149 from [BEAM-9533] Adding tox cloud tests

[github] Flink 1.10 yarn deployment fix (#11146)

[github] [BEAM-9539] Fix copy-pasted comment in load-tests' build.gradle (#11155)

[lcwik] [BEAM-9540] Rename beam:source:runner:0.1/beam:sink:runner:0.1 to

[github] [BEAM-9526] Add missing unmarshalling in top.LargestPerKey. (#11143)

[robertwb] [BEAM-9535] Remove unused ParDoPayload.Parameters.

[github] Merge pull request #11147 from [BEAM-7923] Support dict and iterable

[coheigea] BEAM-8924 - Update Apache Tika to 1.24

[robertwb] [BEAM-9339] Declare capabilities in the Java SDK.

[kawaigin] [BEAM-7923] Change Transform Label Prefix Syntax

[github] Specify return types of window start/end functions explicitly (#11152)

[apilloud] [BEAM-9511] Uncollect takes arbitrary expressions

[lcwik] [BEAM-9339, BEAM-2939] Drop splittable field from proto, add splittable

[apilloud] [BEAM-9515] Add test

[lcwik] [BEAM-4374] Define the protos for a "short" id mechanism for metrics

[robertwb] [BEAM-9339] Add additional Java capabilities.

[kcweaver] [BEAM-9553] Use latest Flink job server image as default.

[jfarr1] [BEAM-9470] fix flaky unit test in :sdks:java:io:kinesis

[relax] fix side-input joins

[relax] fix bug

[github] Merge pull request #11158 from [BEAM-9533] Fixing tox.ini variants

[relax] remove obsolete test

[alex] [BEAM-9035] BIP-1: Typed options for Row Schema and Field

[iemejia] [BEAM-9279] Refactor HBase to disminish relying on Serializable wrappers

[iemejia] [BEAM-9279] Make HBase.ReadAll based on Reads instead of HBaseQuery

[github] [BEAM-9551] Environment PB Pointer cleanup (#11164)

[relax] add javadoc

[iemejia] Move CHANGES template related items into template section

[rohde.samuel] Add dependency comment in streaming cache

[robertwb] Fix and test tuple inputs and outputs.

[lcwik] fixup! Address PR comments.

[github] Merge pull request #11166 from [BEAM-7923] Emit info when capture

[github] fix typo at Python Package name (#11098)

[github]  [BEAM-9552] Bump TestPubsub subscription creation ACK deadline to 60s

[daniel.o.programmer] [BEAM-3301] Perform SDF validation (missing RestrictionTrackers).

[kawaigin] Remove the excessive logging from capturable sources property.

[ehudm] [BEAM-8280] Enable type hint annotations

[github] Merge pull request #11128 from [BEAM-9524] Fix for executing

[lcwik] [BEAM-9430] Update to reflect removal of

[piotr.szuberski] [BEAM-9563] Change ToListCombineFn access level to private

[boyuanz] Add Timer to Elements proto representation.

[robertwb] [BEAM-9340] Plumb requirements through Java SDK.

[robertwb] [BEAM-9340] Populate requirements for Java DoFn properties.

[daniel.o.programmer] [BEAM-3301] Bugfix in DoFn validation.

[robertwb] [BEAM-9558] Add an explicit end field to the data channel protos.

[robertwb] [BEAM-9558] Regenerate go protos.

[robertwb] [BEAM-9558] Produce and respect data channel end bit in runners and

[github] Merge pull request #11153 from [BEAM-9537] Adding a new module for

[kcweaver] [BEAM-9446] Retain unknown arguments when using uber jar job server.

[lcwik] [BEAM-9565] Fix threading issue with WatermarkEstimatorsTest

[relax] add unit tests

[relax] update sql transform

[lcwik] [BEAM-9430] Fix coder sent to Dataflow service for non-portable

[kcweaver] Enable '--option=value' and '-option' syntax.

[github] Merge pull request #10990: [BEAM-9569] disable coder inference for rows

[rohde.samuel] Adds a streaming wordcount integration test

[rohde.samuel] Address leftover styling comments from PR/10892

[github] optionally import grpc (#11187)

[github] [BEAM-9305] Allow value provider query strings in _CustomBigQuerySource

[rohde.samuel] address comments

[github] Merge pull request #11198 from [BEAM-7923] Obfuscates display ids

[github] Merge pull request #11074: Store logical type values in Row instead of

[piotr.szuberski] [BEAM-9507] Fix python dependency check task

[mxm] [BEAM-9573] Correct computing of watermark hold for timer output

[github] [BEAM-7923] Pop failed transform when error is raised (#11174)

[pabloem] Fixing bqtest

[suztomo] google-api-client 1.30.9

[github] [BEAM-9579] Fix numpy logic operators (#11204)

[mxm] [BEAM-9580] Downgrade Flink version to 1.9 for Nexmark and

[github] Merge pull request #11163 from [BEAM-9548] Add better error handling to

[daniel.o.programmer] [BEAM-3301] Adding restriction trackers and validation.

[ehudm] [BEAM-8280] Type hints via annotations snippets

[rohde.samuel] changed data to be less repetitive

[github] Merge pull request #11215 from [BEAM-9601] unbreaking precommits

[pabloem] Starting refactor of FnApiRunner

[pabloem] fixup

[pabloem] Fix lint issues

[pabloem] Creating file with worker handlers

[pabloem] Fixup

[pabloem] Fixing lint. Formatting

[pabloem] Fixup

[jozo.vilcek] [BEAM-9420] Configurable timeout for blocking kafka API call(s)

[piotr.szuberski] [BEAM-9606] Add missing parameters in gradle example

[github] Go changes for model updates. (#11211)

[alex] [BEAM-9605] BIP-1: Rename setRowOption to setOption on Option builder

[robertwb] Add base SDK version to environment capabilities for Python and Java.

[apilloud] [BEAM-9609] Upgrade to ZetaSQL 2020.03.2

[robertwb] [BEAM-9340] Validate pipeline requirements in PipelineValidator.

[pabloem] Renaming method to be more appropriate

[daniel.o.programmer] [BEAM-3301] Fix another bug in DoFn validation, in exec.

[github] fix formatter

[github] [BEAM-8078] is missing a test (#10914)

[rohde.samuel] [BEAM-9601] Skip the streaming wordcount test because it uses a

[github] Update Go Protos (#11230)

[github] add @RequiresTimeSortedInput to (#11228)

[github] Add notes to change log.

[ehudm] [BEAM-8078] Disable test_streaming_wordcount_it

[zyichi] [BEAM-9263] Bump up python sdk dataflow environment major versions

[github] [BEAM-9371] Add SideInputLoadTest to Java SDK (#10949)

[github] [BEAM-7505] Add side input load test to Python SDK  (#11136)

[mxm] [BEAM-9566] Mitigate performance issue for output timestamp watermark

[robertwb] [BEAM-9614] Add SDK id for go.

[ehudm] [BEAM-8078] Disable test_streaming_wordcount_debugging_it

[github] [BEAM-9495] Make DataCatalogTableProvider AutoCloseable (#11116)

[ehudm] [BEAM-8466] Make strip_iterable more strict

[robertwb] [BEAM-4150] Use explicit map for data channel coders.

[robertwb] [BEAM-4150] Don't window PCollection coders.

[ehudm] [BEAM-5422] Document DynamicDestinations.getTable uniqueness requirement

[ehudm] [BEAM-1894] Remove obsolete EagerRunner test

[github] [BEAM-9574] Ensure that instances of generated namedtuple classes can be

[github] Update the range for pyarrow

[boyuanz] Remove TimeSpec from proto

[github] add missing bracket

[alex] [BEAM-9605] BIP-1: Rename setRowOption to setOption on Option builder

[lcwik] [BEAM-4374] Update protos related to MonitoringInfo.

[github] Update the Go SDK roadmap for April 2020 (#11239)

[github] Merge pull request #10883: [BEAM-9331] Add better Row builders

[github] [BEAM-8292] Portable Reshuffle for Go SDK (#11197)

[robertwb] Side input signals for legacy worker.

[lostluck] accept generated metrics

[github] Use split instead of rsplit

[lostluck] Remove mType and move type urns to urnToType

[lostluck] add missing pcollection payload

[github] [BEAM-9557] Fix timer window boundary checking (#11252)

[github] Update documentation

[echauchot] [BEAM-5980] Change load-tests build to include spark-sql for spark

[echauchot] [BEAM-9436] avoid one flatmap step and a KV creation per element by

[jozsi] Update Jet version to 4.0

[apilloud] [BEAM-9512] Map anonymous structs to schema

[kawaigin] [BEAM-7923] Fix datatable on notebook reloading

[kyoungha] [BEAM-9325] Added Proper Write Method in UnownedOutputStream

[kyoungha] fix format warning

[valentyn] Fix a Py2/3 incompatibility in profiler.

[kcweaver] [BEAM-9638] Strengthen worker region & zone options tests.

[samuelw] [BEAM-9399] Change the redirection of System.err to be a custom

[kyoungha] [BEAM-9325] reflect comment : inline testing methods

[robinyqiu] Clean up code in ZetaSQLDialectSpecTest

[boyuanz] [BEAM-9454] Add Deduplication PTransform

[robertwb] [BEAM-9577] Rename the Artifact{Staging,Retrieval}Service.

[robertwb] [BEAM-9577] Define the new Artifact{Staging,Retrieval}Service.

[robertwb] [BEAM-9577] Regenerate protos.

[jozsi] Update Jet Runner web page with info about 4.0

[jozsi] Add Beam-Jet compatibility table

[robertwb] [BEAM-9577] Implement the new Artifact{Staging,Retrieval}Services in

[samuelw] Fix missing test import

[kyoungha] [BEAM-9325] reflect comment : Fix JAXBCoder + change test

[chamikara] Refactors X-Lang test pipelines.

[robertwb] [BEAM-9340] Populate requirement for timer families.

[kcweaver] [BEAM-9199] Require Dataflow --region in Python SDK.

[kcweaver] Add --region to tests where needed.

[kcweaver] [BEAM-9199] Require --region option for Dataflow in Java SDK.

[kcweaver] Add --region to Java GCP tests.

[pabloem] [BEAM-9608] Increase reliance on Context Managers for FnApiRunner

[pabloem] Revert "Merge pull request #11104 from y1chi/update_tornado_test"

[daniel.o.programmer] [BEAM-9642] Create runtime invokers for SDF methods.

[lcwik] [BEAM-9668] Disable tests till Dataflow containers are updated.

[github] [BEAM-9652] Ensure that the multipartition write sets the correct coder

[github] [BEAM-8889]add experiment flag use_grpc_for_gcs (#11183)

[robertwb] [BEAM-9322] [BEAM-1833] Better naming for composite transform output

[ameihm] [BEAM-9476] KinesisIO retry LimitExceededException

[github] [BEAM-7923] An indicator of progress in notebooks (#11276)

[robertwb] [BEAM-9577] Add dependency information to provision info.

[robertwb] Update go protos.

[lcwik] [BEAM-9677] Fix path -> url typo in ArtifactUrlPayload

[lcwik] [BEAM-9562] Update missed TimerSpec conversion in Go SDK

[kcweaver] Fix DataflowRunnerTest.

[kcweaver] Fix more Java unit tests missing --region.

[github] [BEAM-9667] Allow metrics in DoFn Setup (#11287)

[amaliujia] add 2.20.0 blog post

[kcweaver] Add --region to DF streaming example tests.

[github] [BEAM-9624] Adds Convert to Accumulators operator for use in combiner

[github] Fix minor typo

[github] Fix minor typo

[github] Merge pull request #11290: [BEAM-9670] Fix nullability widening in

[iemejia] [BEAM-9686] Get default TmpCheckpointDir value from PipelineOptions

[github] [BEAM-4374] Short IDs for the Python SDK (#11286)

[spoorti] [BEAM-9660]: Add an explicit check for integer overflow.

[github] [BEAM-9136]Add licenses for dependencies for Python (#11067)

[boyuanz] Populate source data from SDF

[boyuanz] Update Timer encoding

[mxm] [BEAM-9645] Fix premature removal of Docker container and logs

[alex] [BEAM-9044] Protobuf options to Schema options

[kcweaver] Add unit tests for get_default_gcp_region

[kcweaver] Add --region to Dataflow runner webpage.

[mxm] [BEAM-8201] Cleanup FnServices from DockerEnvironmentFactory and

[kcweaver] lint

[kcweaver] Add --region to more Java tests and examples.

[kcweaver] Add --region to more Python tests and examples.

[kcweaver] format

[robertwb] [BEAM-9577] Update container boot code to stage from dependencies, if

[rohde.samuel] Change delimeter to a dash as it is a reserved symbol in Windows

[valentyn] Fixes platform-dependent assumptions in

[valentyn] Switches a test helper to a Py3-version thereof.

[github] Apply suggestions from code review

[robertwb] Use pointer recievers.

[aldaircr] Change: Fixing typos on javadoc

[robertwb] Attempt to stage resources via new API in portable runner.

[robertwb] ResolveArtifact -> ResolveArtifacts

[robertwb] Regenerate protos.

[veblush] Upgrades gcsio to 2.1.2

[github] Merge pull request #11259: Use attachValues in SQL

[alex] Add Beam Schema Options to changelog

[iemejia] [website] Update information about Beam's LTS policy

[alex] [BEAM-9704] Deprecate FieldType metadata

[eekkaaadrian] [BEAM-9705] Go sdk add value length validation checking on write to

[kcweaver] Remove unrecognized --region option from non-DF tests.

[robertwb] [BEAM-9618] Add protocol for requesting process bundle descriptors.

[robertwb] [BEAM-9618] Update Python to support process bundle descriptor fetching.

[robertwb] [BEAM-9618] Java FnApiClient support for process bundle descriptor

[github] [BEAM-8019] Python SDK support for cross-langauge pipelines in Dataflow.

[robertwb] Typo fix.

[github] remove nose (#11307)

[lcwik] [BEAM-4374, BEAM-6189] Delete and remove deprecated Metrics proto

[ecapoccia] [BEAM-9434] Improve Spark runner reshuffle translation to maximize

[github] [BEAM-9685] remove Go SDK container from release process (#11308)

[kcweaver] [BEAM-9716] Alias zone to worker_zone and warn user.

[github] Merge pull request #11226: [BEAM-9557] Fix timer window boundary

[github] Merge pull request #11244 from [BEAM-3097] _ReadFromBigQuery supports

[pabloem] [BEAM-9691] Ensuring BQSource is avoided on FnApi

[pabloem] [BEAM-9715] Ensuring annotations_test passes in all

[github] Name the pipeline_v1 proto import

[github] Update materialize_test.go

[ankurgoenka] [BEAM-9707] Hardcode Unified harness image for fixing dataflow VR 2

[crites] Updates documentation for WINDOWED_VALUE coder.

[rohde.samuel] Fix flaky interactive_runner_test

[github] Merge pull request #11205 [BEAM-9578] Defer expensive artifact

[robertwb] Update go protos.

[github] Fix some Go SDK linter/vet warnings. (#11330)

[robertwb] [BEAM-9577] Plumb resources through Python job service and runner.

[robertwb] [BEAM-9618] Pull bundle descriptors for Go.

[github] [BEAM-9529] Remove datastore.v1, googledatastore (#11175)

[github] Update session.go

[github] Update stage.go

[github] Update server_test.go

[github] Update materialize.go

[github] Update materialize_test.go

[github] Update stage_test.go

[github] Update artifact.go

[github] Update provision.go

[github] Update retrieval.go

[github] Update staging.go

[github] Update translate.go

[github] Update datamgr.go

[github] Update datamgr_test.go

[github] Update logging.go

[github] Update logging_test.go

[github] Update monitoring.go

[github] Update session.go

[github] Update statemgr.go

[github] Update statemgr_test.go

[github] Update replace.go

[github] Update replace_test.go

[github] Update provision.go

[github] Update execute.go

[github] Update job.go

[github] Update translate.go

[github] Update translate.go

[github] Update job.go

[github] Update materialize.go

[mxm] [BEAM-9580] Allow Flink 1.10 processing timers to finish on pipeline

[kamil.wasilewski] [BEAM-9721] Add --region to Dataflow-based load tests

[kamil.wasilewski] [BEAM-9721] LoadTestConfig: handle --region parameter and put default

[github] [BEAM-9147] Add a VideoIntelligence transform to Java SDK (#11261)

[mxm] Revert "[BEAM-9580] Downgrade Flink version to 1.9 for Nexmark and

[kcweaver] [BEAM-9714] [Go SDK] Require --region flag in Dataflow runner.

[github] Update translate.go

[mxm] [BEAM-9557] Fix strings used to verify test output

[github] Update session.go

[github] Update materialize_test.go

[mxm] [BEAM-9596] Ensure metrics are available in PipelineResult when the

[samuelw] Ensure that empty messages are not flushed to handler.

[crites] Uses iterable coder for windows and copies all of timestamp encoding

[github] Update session.go (#11352)

[github] [BEAM-9618] Java SDK worker support for pulling bundle descriptors.

[chamikara] Adds nose back under packages needed for testing.

[robertwb] [BEAM-9618] Mark push registration as deprecated.

[github] [Beam-9063]update documentation (#10952)

[kcweaver] [BEAM-9726] [py] Make region optional for non-service Dataflow.

[kcweaver] [BEAM-9726] [java] Make region optional for non-service runner.

[github] [BEAM-9550] Increase JVM Metaspace size for the TaskExecutors. (#11193)

[github] [BEAM-9721]Conditionally add Dataflow region to Dataflow-based

[michael.jacoby] [BEAM-9647] fixes MQTT clientId to long

[lcwik] [BEAM-4374] Fix missing deletion of metrics.

[github] [BEAM-8280] Document Python 3 annotations support (#11232)

[github] [BEAM-9731] Include more detail in passert.Equals errors. (#11359)

[github] [BEAM-9085] Fix performance regression in SyntheticSource on Python 3

[amaliujia] add a known issue

[samuelw] [BEAM-9651] Prevent StreamPool and stream initialization livelock

[boyuanz] [BEAM-9562, BEAM-6274] Fix-up timers to use Elements.Timer proto in data

[robertwb] Allow unset write threshold for state backed iterable coder.

[github] Revert "[BEAM-9651] Prevent StreamPool and stream initialization

[ankurgoenka] [BEAM-9735] Adding Always trigger and using it in Reshuffle

[samuelw] [BEAM-9651] Prevent StreamPool and stream initialization livelock

[github] [BEAM-9727] Automatically set required experiment flags for dataflow

[github] Update to add a method to specify container image

[huangry] Include user distritribution into metric-dedicated validate runner test.

[huangry] update inline comment.

[huangry] To allow only assert on committed metrics, bypassing the attempted ones.

[huangry] Use distribution matcher for testing.

[kcweaver] Moving to 2.22.0-SNAPSHOT on master branch.

[kamil.wasilewski] [BEAM-8671] Migrate Load Tests to Python 3.7

[michal.walenia] [BEAM-9734] Revert #11122

[sebastian.graca] [BEAM-9439] Return split instead of total backlog size

[github] Add --region to changelog

[pabloem] Fix from_container_image call

[ankurgoenka] TOIL: Update Unified worker image

[boyuanz] [BEAM-9562] Update Element.timer, Element.Timer to Element.timers and

[robertwb] [BEAM-9577] New artifact staging and retrieval service for Java.

[robertwb] [BEAM-9577] Multi-threaded artifact staging service backend.

[alxavier] [BEAM-9739]Retry SpannerIO write on Schema change

[robertwb] Comments and clarification.

[github] [BEAM-9443] support direct_num_workers=0 (#11372)

[APryor] Correct a typo in documentation

[chamikara] Updates Dataflow stateful DoFn setup to support external transforms

[github] [BEAM-9738] Update dataflow to setup correct docker environment options.

[github] [BEAM-9136]Add licenses for dependencies for Java (#11243)

[kcweaver] [BEAM-9744] Add missing region option to py perf tests.

[lcwik] [BEAM-9562] Fix output timestamp to be inferred from scheduled time when

[kcweaver] [BEAM-9744] Remove --region option from SQL tests.

[lcwik] [BEAM-2939] Update unbounded source as SDF wrapper to resume

[pabloem] Fixing type names for BQ Avro Tools

[github] Merge pull request #11389 from Refactor the BCJ and capture controls to

[github] [BEAM-i9751] upgrade zetasql to 2020.04.1 (#11410)

[xhan] Documentation bug fix for FlatMapElements#via() SimpleFunction in the

[github] [BEAM-9650] Add PeriodicImpulse Transform and slowly changing side input

[github] [BEAM-7923] Screendiff Integration Tests (#11338)

[lcwik] fixup! Fix spotbugs warning

[kcweaver] [BEAM-9756] Nexmark: only use --region in Dataflow.

[github] [BEAM-9642] Add SDF execution units. (#11327)

[lcwik] [BEAM-9577] Fix test to create urls from paths which are compatible with

[github] [BEAM-9136] reduce third_party_dependencies size (#11416)

[thw] Fix py37-lint

[thw] Maven compatible publish repository authentication via settings.xml

[github] [BEAM-9746] check for 0 length copies from state (#11413)

[pabloem] Removing underscore from _ReadFromBigQuery to make it external. It

[daniel.o.programmer] [BEAM-9642] Fix infinite recursion.

[kamil.wasilewski] Remove outdated doc for ReadFromBigQuery transform

[ehudm] [BEAM-9119] Disable flaky test

[github] [BEAM-8889] add gRPC suport in GCS connector (behind an

[amaliujia] fixup! update 2.20.0 date

[github] [BEAM-9729, BEAM-8486] Runner-side bundle registration cleanup. (#11358)

[kcweaver] [BEAM-9765] Exclude module-info.class from vendored Calcite.

[robertwb] [BEAM-9577] Small fixes to portable runner staging.

[robertwb] [BEAM-9768] Gradle command for Python ULR.

[github] Add new release 2.20.0 to beam website (#11285)

[pabloem] [BEAM-9608] BundleManagers use BundleContextManager for configuration

[jkw] Fix typo

[pabloem] [BEAM-9769] Ensuring JSON is the default export format for BQ sink

[github] Merge pull request #11151 from [BEAM-9468]  Hl7v2 io

[echauchot] Add GBK load tests script for spark structured streaming runner

[echauchot] Add CoGBK load tests script for spark structured streaming runner

[echauchot] Add Combine load tests script for spark structured streaming runner

[echauchot] Add Pardo load tests script for spark structured streaming runner

[nielm] Remove spurious error message.

[ehudm] [BEAM-6860] Fix iobase non-global windows bug

[github] [BEAM-9678] Create Go SDK introduction kata (#11340)

[rohde.samuel] Add a timeout to the TestStream GRPC and fix the Streaming cache timeout

[github] Merge pull request #11436: Document Beam Schemas (forked PR)

[github] [BEAM-9648]: DirectRunner should return null on timeout (#11275)

[ehudm] [BEAM-9737] Don't use docker create option -u

[github] [BEAM-5379] Ignore go.sum files in RAT checks

[skoonce] [BEAM-9773]: Update Dataflow Debug Capture to use Google API client

[github] [BEAM-5605, BEAM-2939] Add support for FnApiDoFnRunner to handle split

[pabloem] [BEAM-9639] Saving side inputs after producer execution, not before

[pabloem] [BEAM-9639] Separate Stage and Bundle execution. Improve typing

[github] [BEAM-9674] Don't specify selected fields when fetching BigQuery table

[github] [BEAM-9650] Cleanup documentation on side inputs patterns (#11415)

[github] [BEAM-8646] Fix external environment on OS X as well. (#11431)

[github] [BEAM-9743] Fix TFRecordCodec to try harder to read/write (#11397)

[github] Updates and clarifications on type safety. (#11449)

[kcweaver] [BEAM-9600] Get metrics in Flink uber jar job server.

[kcweaver] [BEAM-9600] Add a test for Flink REST metrics.

[valentyn] Renames run_test -> run_scenario so that nose test runner does not think

[huangry] Fix user distribution metric test by using less restricting value.

[github] Log job id similar to Java and Go

[robertwb] [BEAM-6960] More sophisticated job server starting and stopping.

[robertwb] Abort pipeline if docker containers die unexpectedly.

[robertwb] [BEAM-6960] Go ULR validates runner tests.

[huangry] Apply yapf fix.

[github] [BEAM-3836] Enable dynamic splitting/checkpointing within the Java SDK

[github] [BEAM-8872] Support split at fraction for OffsetRangeTracker (#11418)

[iemejia] [BEAM-9569] Fix BeamSqlPojoExample logRecords method to not require Row

[github] Display correct package in help message

[mxm] [BEAM-9733] Always let ImpulseSourceFunction emit a final watermark

[kawaigin] add display:block to datatable so that the columns can have various

[michal.walenia] [BEAM-9147] Add @Experimental annotation to VideoIntelligence related

[michal.walenia] Add @Experimental to the package-info as well

[iemejia] [BEAM-9285] Add Postcommit ValidatesRunner CI Job for Flink on Java 11

[skoonce] Remove all Dataflow google-api-client-jackson(1) dependencies

[huangry] Replace all explicit validation.

[skoonce] Missed cleanup

[github] Add release date for 2.20.0

[github] [BEAM-9692]: Make CombineValues portable (#11335)

[aiyangar] Changed sns writes to async

[robertwb] lint

[github] [BEAM-9788] Add RetainDockerContainers flag support to Go SDK. (#11467)

[github] [BEAM-9775] Add Go support for SDF StandardRequirements. (#11443)

[github] [BEAM-9496] Add to_dataframe and to_pcollection APIs. (#11264)

[ankurgoenka] [BEAM-9791] Add precommit for dataflow runner v2

[apilloud] [BEAM-9662][BEAM-9663] Reject Null Limit and Offset

[boyuanz] Fix SplittableParDoNaiveBounded DoFnInvoker.

[shoaib.zafar] update release 2.20.0 blog post and added contributors name

[mxm] [BEAM-9794] Reduce state cells needed for BufferingDoFnRunner

[sniemitz] [BEAM-9795] Support custom avro DatumWriters when writing to BigQuery

[github] Revert "[BEAM-8131] Provide Kubernetes setup for Prometheus (#9482)"

[github] Merge pull request #11466 from [BEAM-9787] Clear error message on UW +

[pabloem] [BEAM-9639][BEAM-9608] Addressing review comments.

[github] [BEAM-9764] multi threading & optional pulling (#11428)

[github] Revert "[BEAM-9014] CachingShuffleBatchReader use bytes to limit cache

[github] [BEAM-9789] Fix lock error. Add test. (#11468)

[github] [BEAM-7981] Correct ParDo(callable) test (#11446)

[apilloud] [BEAM-9514] Ensure nullability passes through sum

[rohde.samuel] [BEAM-9692] Replace the apply_read with a PTransformOverride

[kamil.wasilewski] [BEAM-8790] Kubernetes deployment for InfluxDB

[kamil.wasilewski] [BEAM-8790] Local deployment of InfluxDB with docker-compose.yml

[github] [BEAM-6597, BEAM-6467] Add support for reporting monitoring infos to the

[yoshiki.obata] [BEAM-9755] changed MAKE_ADD stack pop order according to Python version

[github] Merge pull request #11489 [BEAM-9577] Ensure required directories exist

[github] [BEAM-9642] Enabling SDF exec runners and fixing small bug. (#11474)

[robertwb] Conditionally cleanup tempdir.

[github] [BEAM-9717] update future version to 0.18.2 for Python 3.8 (#11494)

[github] [BEAM-9577] Use new artifact API to stage go artifacts. (#11490)

[kcweaver] [BEAM-9807] [BEAM-9808] Add missing region to Dataflow examples.

[github] Force output names to be unicode-compatible on Python 2. (#11485)

[github] Make tolerate directory already existing (#11501)

[github] [BEAM-9778] remove temp directory & add docker-pull-licenses to release

[lcwik] [BEAM-2939] Expose HasProgress interface for restriction trackers and

[lcwik] [BEAM-2939] Add the ability for SDK harness runners to provide

[lcwik] [BEAM-2939] Integrate progress reporting for splittable dofns.

[lcwik] [BEAM-2939] Fold Sizes sub-interfaces into RestrictionTracker

[lcwik] [BEAM-2939] Drop HasSize in favor of using

[github] [BEAM-8603] Add Python SqlTransform (#10055)

[kcweaver] Missed a few spots.

[echauchot] [BEAM-5192] move metadata serialization configuration code to

[mxm] [BEAM-9733] Make up for timers set while processing the bundle

[robertwb] Use futures, better error handling.

[github] [BEAM-9701] Increments fastavro version range upper bound to 0.24.

[pabloem] [BEAM-9812] Fixing bug causing pipelines requiring temp tables to not

[robertwb] [9577] Attempt to stage artifacts via new API on Java.

[ehudm] [BEAM-7405] Workaround for bad Docker config

[boyuanz] [BEAM-8871] Support trySplit for ByteKeyRangeTracker

[robertwb] [BEAM-9813] Replace outputs with respect to the correct tag.

[tweise] [BEAM-9811] Nightly snapshot publish error

[github] [BEAM-9775] Adding Go SDF example, adjusting GetProgress signature.

[kamil.wasilewski] [BEAM-9761] Disable auto builds for tests that are currently failing.

[tweise] [BEAM-9811] Copy pom for all publishMavenJavaPublication tasks

[github] add licenses and license types for zetasql packages (#11516)

[github] Added a batch example with covid tracking data for interactive notebook.

[kcweaver] Only add --region for Dataflow runner.

[github] Fix invalid formatting specifier in verify

[github] [BEAM-9488] Ensure we pass through PCollection ids instead of attempting

[iemejia] [BEAM-9820] Upgrade Flink 1.9.x to 1.9.3

[github] [BEAM-9819] Relaxes the version range for httplib2 to <0.16.0. (#11522)

[coheigea] BEAM-9826 - Update Tika to 1.24.1

[mxm] [BEAM-9827] Ensure minimum watermark hold is computed across all keys

[github] Merge pull request #11154: [BEAM-1819] Key should be available in

[mxm] [BEAM-6661] Properly close channel for external environment

[mxm] [BEAM-6661] Avoid FileNotFoundException when no files have been staged

[mxm] [BEAM-6661] Avoid printing empty lines in subprocess job server

[apilloud] [BEAM-9658] Plumb through WITH OFFSET

[github] Updates Dataflow Python containers to 20200423 (#11535)

[bhulette] Fix JavaSDK typo

[bhulette] Add Python SqlTransform to

[github] [BEAM-9779] Patch HL7v2IOWriteIT Flakiness (#11450)

[github] virtualenv (#11502)

[kcweaver] [BEAM-9833] Add .asf.yaml file.

[kcweaver] Send issues to github@

[kcweaver] Add license header.

[kcweaver] [BEAM-9758] Use quotes in echo statements to prevent * expansion.

[github] Merge pull request #11546 from [BEAM-9832] Ensuring tags are not added

[david.moravek] [BEAM-9824] Do not ignore chained Reshuffles on flink batch runner.

[github] [BEAM-9443] update document about direct_num_workers=0 for FnApi

[robertwb] Use logging to reduce verbosity of license checking when all goes well.

[lcwik] [BEAM-9846] Clean up references to unused native BQ source/sink

[github] Speedup on FnApiRunner start up time (#11543)

[github] All Python DirectRunners call setup/teardown on DoFns (#11547)

[github] [BEAM-9815] Correct go integration test if clauses.  (#11524)

[relax] Support schemas in state API

[github] [BEAM-7923] Support docker screendiff integration tests (#11507)

[robertwb] [BEAM-9802] Customization of automatically started services.

[jhnmora000] [BEAM-9418] Support ANY_VALUE aggregation functions

[github] [BEAM-9147] Make VideoIntelligence use PTransform on user-facing API

[bhulette] remove PipelineRunner.fromOptions call

[dcavazos] [BEAM-9771] Fix notebook github URLs

[mxm] [BEAM-2128] Remove dependency on PipelineRunner#fromOptions to stage

[bhulette] Modify PipelineTest to check if exception is thrown on Pipeline run

[bhulette] Call FileSystems.setDefaultPipelineOptions in

[github] Fix typo in go installation link. (#11445)

[rohde.samuel] move redundant testing pieces to function

[github] [BEAM-9561] Add a framework for running pandas doctests with beam

[echauchot] [BEAM-8025] Increase the number of retrials en retrial delay in case of

[github] [BEAM-9720] Add Aws Http client configurations capability. (#11341)

[mxm] [BEAM-9801] Pass in fire timestamp and pane info to timer callback

[mxm] [BEAM-9733] Repeatedly fire in batch mode until there are no more timers

[github] skip pulling licenses by default (#11548)

[boyuanz] Update container version

[github] [BEAM-8078] is missing a test (#11568)

[github] Merge pull request #11538 from [BEAM-9831] Improve UX for HL7v2IO

[kcweaver] [BEAM-9860] Require job_endpoint when using PortableRunner.

[kcweaver] Replace portableWordCount tests with portableWordCountFlinkRunner.

[robertwb] Guard grpc import.

[sjvanrossum] [BEAM-9776] Fixes filesystem not found error

[robertwb] Test simplification.

[lcwik] [BEAM-8944] Improve UnboundedThreadPoolExecutor performance

[lcwik] fixup! Fix lint

[kcweaver] Refactor timer firing in Flink & Spark portable batch.

[kcweaver] [BEAM-9801] Fire timers set within timers in Spark.

[mxm] [BEAM-9841] Support finite wait on PortableRunner's PipelineResult

[github] Merge pull request #11523 from [BEAM-8414] Cleanup Python codebase to

[pawel.pasterz] Update error prone

[pawel.pasterz] Aling EP core and annotations versions

[mxm] [BEAM-9874] Support clearing timers in portable batch mode (Spark/Flink)

[tudor.marian] Do not attempt to estimate sizes of group-by-key values behind a key;

[echauchot] [BEAM-8025] Trace caught NoHostAvailableException and

[tudor.marian] remove comments

[github] [BEAM-9870] Always generate Dataflow-compatible FnApi protos. (#11593)

[tudor.marian] Make BatchGroupAlsoByWindowViaIteratorsFn extend the

[tudor.marian] revert changes to grouping shuffle reader and iterable like coder

[aiyangar] [BEAM-8542] V2 async changes for SNSIO

[tudor.marian] fix merge conflicts

[tudor.marian] remove obsolete class

[github] Go fmt changes. (#11576)

[apilloud] [BEAM-9661] Fix ORDER BY with LIMIT

[kcweaver] [BEAM-9875] Use FlinkRunner instead of PortableRunner in cross-language

[apilloud] [BEAM-9659][BEAM-9657][BEAM-9664] Reject unsupported unnest joins

[daniel.o.programmer] [BEAM-9799] Automated validation for RTrackers.

[daniel.o.programmer] [BEAM-9643] Adding Go SDF Documentation.

[github] [BEAM-9880] fix cannot touch issue (#11606)

[kcweaver] Remove runner check in xlang wordcount.

[daniel.o.programmer] [BEAM-9643] Adjusting signature for TrySplit.

[github] Merge pull request #11560 from [BEAM-9886] Auto-inferring project for

[github] [BEAM-9650] Add PeriodicSequence generator. (#11477)

[je.ik] Disable two flaky tests (BEAM-8035, BEAM-9164)

[github] Pass pipeline options from caller to expansion service (#11574)

[je.ik] [BEAM-9888] Drop data based on input watermark in

[robertwb] Move job server to its own module.

[robertwb] [BEAM-9845] Stage artifacts over expansion service.

[robertwb] [BEAM-9577] Resolve dependencies in Java expansion service.

[robertwb] Start artifact service in expansion service driver. 0;256;0c# Please

[robertwb] [BEAM-7746] Enable mypy type checking for Beam Python code.

[kcweaver] [BEAM-9908] Fix Python build failures in release script.

[github] [BEAM-9840] Support for Parameterized Types when converting from

[kcweaver] Update instructions in release guide.

[iemejia] [BEAM-9836] Exclude Spark runner from UsesKeyInParDo tests

[robertwb] Fix thread local to be initialized on every thread.

[alxavier] [BEAM-9911]Replace SpannerIO.write latency counter to distribution

[github] un-duplicate _BEAM_SERVICES

[github] Update sdks/python/mypy.ini

[github] Remove a bunch of spurious warnings in tests. (#11621)

[github] [BEAM-8134] Grafana dashboards for Load Tests and IO IT Performance

[github] [BEAM-8133] Push ioit tests metrics to influxdb

[mxm] [BEAM-8742] Add stateful and timely processing benchmarks

[mxm] [BEAM-9900] Remove shutdownSourcesOnFinalWatermark flag

[kamil.wasilewski] [BEAM-6710] Add landing page with links to relevant dashboards

[github] [BEAM-9913] remove license dir if exists (#11630)

[github] Merge pull request #11624 from [BEAM-9767] Make streaming_wordcount use

[github] [BEAM-9731] passert.Equals: sort output strings for easier reading

[github] Revert "Pass pipeline options from caller to expansion service (#11574)"

[kcweaver] [BEAM-4782] Remove workaround in Python multimap tests.

[klk] Remove tests of Java on Dataflows Java Runner Harness

[github] [Beam-9679] Add Core Transforms section / Map lesson to the Go SDK katas

[amaliujia] [BEAM-9929] Support UNNEST(array_column) in ZetaSQL.

[mxm] [BEAM-9930] Beam Summit Digital 2020 announcement on blog

[mxm] Correct URL in blog post

[matthias.baetens] Minor (markup) changes

[kamil.wasilewski] [BEAM-8132, BEAM-8133] Apply InfluxDB pipeline options in Load Tests and

[kamil.wasilewski] [BEAM-8132, BEAM-8133] Assume no trailing slash after a hostname

[iemejia] [BEAM-2530] Compile and run tests on java 11 for Precommit portability

[github] [BEAM-9883] Refactor SDF test restrictions. (#11605)

[kcweaver] [BEAM-9835] [Portable Spark] Broadcast a PCollection at most once.

[chambers] Add a BeamJava test with Flatten with different input and output Coders

[github] [BEAM-3288] Add suggested fix to error message (#11622)

[jferriero] check lastAttemptedOffset non-null

[zyichi] [BEAM-9940] Set timer family spec for TimerDeclarations in dataflow

[robertwb] [BEAM-9935] Respect allowed split points in Python.

[mxm] [BEAM-9930] Correct authors of beam summit digital blog post

[mxm] [BEAM-9930] Add image to Beam Summit blog post

[github] Add License file for the Go SDK  for

[sniemitz] Allow users of AvroIO to specify a custom DatumReader implementation

[pawel.pasterz] [BEAM-2530] PreCommit Dataflow examples - test compiled with java 11

[mxm] [BEAM-9947] Store updated ParDoPayload for length-prefixed timer coders

[kamil.wasilewski] Fix InfluxDB credentials binding

[mxm] [BEAM-9947] Add unit test to check length prefixing of timer key

[robertwb] More comments.

[rohde.samuel] Return empty iterator on empty cache

[rohde.samuel] Change TestStreamImpl to a producer/consumer pattern

[borzoo.esmailloo] [BEAM-9216] Upgrade katas gradle to version 5.2.1

[github] [BEAM-9887] Throw IllegalArgumentException when building Row with

[klk] [BEAM-9952] Revert "remove PipelineRunner.fromOptions call"

[github] Merge pull request #11637 from Waiting for BQ Query and Export jobs for

[robertwb] Make grpc-requiring class definition optional.

[github] [BEAM-9884] Add options to SqlTransform for specifying planner (#11647)

[daniel.o.programmer] [BEAM-9642] Adding Go SDF fallback for unexpanded SDFs.

[kamil.wasilewski] Fix metrics reporting in HDFS IO IT tests

[mxm] [BEAM-9963] Fix coder type in failing ParDo streaming load test

[github] Merge pull request #11567: [BEAM-8132] Report Python metrics to InfluxDB

[mxm] [BEAM-9164] Re-enable UnboundedSourceWrapper#testWatermarkEmission test

[kamil.wasilewski] Fix InfluxDB measurement names to match those in Grafana dashboards

[github] [BEAM-9945] Report data channel progress via a designated counter.

[kawaigin] Use csv reader instead of split to read csv data.

[github] [BEAM-9577] Update Java Runners to handle dependency-based artifact

[github] Merge pull request #11673 from [BEAM-9967] Adding support for BQ labels

[github] [BEAM-9622] Add Python SqlTransform test that joins tagged PCollections

[github] [BEAM-9959] Root Transform fixes (#11686)

[mxm] [BEAM-9966] Use dataproc staging bucket for checkpoints directory

[tysonjh] Disambiguate method filter for toArray.

[pabloem] Revert "Merge pull request #11673 from [BEAM-9967] Adding support for BQ

[thw] Bump Flink 1.10 version

[kcweaver] [BEAM-9975] Log invalid dictionary after unsuccessful struct conversion.

[github] [BEAM-9945] Ensure that the read index represents the number of fully

[github] [BEAM-9942] reduce groovy code duplication in python test-suites

[boyuanz] [BEAM-9935] Respect allowed split points in Java

[apilloud] [BEAM-9522] Reject FULL JOIN ON false

[pabloem] [BEAM-9967] Adding support for BQ labels on Query/Export jobs. (Roll

[github] Switch to %s

[github] [BEAM-9001, BEAM-6327] Ensure that all transforms (except for required

[github] [BEAM-9928] Add qualified import name to task descriptors (#11690)

[mxm] [BEAM-9966] Remove unnecessary Reshuffles

[robinyqiu] Support ZetaSQL DATE type as a Beam LogicalType

[omarismail] [BEAM-9964] Setting --workerCacheMB to make its way to the

[github] [BEAM-9634] Add natural language analysis transform (#11611)

[github] [BEAM-9430] Fixes the bounds of initial watermark set to estimators

[kcweaver] [BEAM-9993] Add option defaults for Flink Python tests.

[github] [BEAM-9941] Added a test of a GBK followed by a Flatten with an unknown

[github] Clarify pubsub IO comment about timestamps (#11672)

[omarismail] [BEAM-9964] Added testMaxWeight method in WIndmillStateCacheTest as per

[iemejia] [BEAM-9833] Add yamllint config

[iemejia] [BEAM-9833] Fix .asf.yaml issues, sort labels and disable rebase button

[github] Clarifies an error message in Katas to explain what is actually wrong.

[github] Merge pull request #11210 from [BEAM-8949] SpannerIO integration tests

[github] [BEAM-9876] Migrate the Beam website from Jekyll to Hugo to enable

[github] Update the range for pyarrow to qualify pyarrow 0.17.x (#11699)

[yifanzou] Run Inventory job twice a day at 6 and 18.

[github] Update release notes link for 2.20.0

[jiangxuehua1213] [BEAM-9136] Update doc about docker image license pulling

[kamil.wasilewski] Make Post-Commits status dashboard compatible with the latest Grafana

[] Modified the codeblock background

[mxm] [BEAM-9966] Propagate the keys to the load generator correctly

[github] [BEAM-2939] Ensure that we don't checkpoint the initial restriction as

[mxm] [BEAM-6733] Add pipeline option to flush bundle data before

[github] support isRelease tag for docker build command & update release guide

[github] [BEAM-1866] Enable validation of PAssert counts in Java validates runner

[daniel.o.programmer] [BEAM-9951] Creating a synthetic source for the Go SDK.

[daniel.o.programmer] [BEAM-9951] Creating a synthetic step for the Go SDK.

[github] Merge pull request #11725: [BEAM-10015] Fix output timestamp on dataflow

[hsuryawirawan] Remove all answer placeholder checks as they can be confusing at times

[hsuryawirawan] Update course in Stepik

[ihr] [BEAM-10018] Fix timestamps in windowing kata

[ihr] [BEAM-10018] Kata failing due to failed parsing

[github] [BEAM-2939] Ensure that we update the watermark even when no elements

[ihr] Remove unused import

[ihr] Add missing dependency

[ihr] Fix member variable name in Kata documentation

[ihr] Fix placeholder location

[github] [BEAM-2530] Implement Zeta SQL precommit compile tests and run on java

[pawel.pasterz] [BEAM-2530] Sql precommit compile and run on java 11

[github] Python3 fix - convert dict.keys() to list before indexing (#11733)

[github] Updates google-apitools and httplib2 (#11726)

[github] [BEAM-9964] Update  (#11743)

[github] [BEAM-9577] Artifact v2 support for uber jars. (#11708)

[nielm] Populate all SpannerIO batching parameters in display data.

[nielm] Fix capitalization, clarify descriptions

[nielm] fix capitalization, clarify description Grouped

[nielm] Refactor to extract single method for popuplating displayData

[github] [BEAM-9339] Ensure that Dataflow's pipeline proto also contains the

[github] [BEAM-9363] TUMBLE as TVF (#10946)

[sruthi.sreekumar] Updated Flink runner version in NEXMark documentation

[kcweaver] Upgrade Dataflow Python container versions.

[github] [BEAM-9383] Staging Dataflow artifacts from environment (#11039)

[github] [BEAM-1589] Added @OnWindowExpiration annotation. (#11350)

Started by timer
Running as SYSTEM
[EnvInject] - Loading node environment variables.
Building remotely on apache-beam-jenkins-1 (beam) in workspace <>
No credentials specified
Wiping out workspace first.
Cloning the remote Git repository
Cloning repository
 > git init <> # timeout=10
Fetching upstream changes from
 > git --version # timeout=10
 > git fetch --tags --progress +refs/heads/*:refs/remotes/origin/*
 > git config remote.origin.url # timeout=10
 > git config --add remote.origin.fetch +refs/heads/*:refs/remotes/origin/* # timeout=10
 > git config remote.origin.url # timeout=10
Fetching upstream changes from
 > git fetch --tags --progress +refs/heads/*:refs/remotes/origin/* +refs/pull/${ghprbPullId}/*:refs/remotes/origin/pr/${ghprbPullId}/*
 > git rev-parse origin/master^{commit} # timeout=10
Checking out Revision 03d99dfa359f44a29a772fcc8ec8b0a237cab113 (origin/master)
 > git config core.sparsecheckout # timeout=10
 > git checkout -f 03d99dfa359f44a29a772fcc8ec8b0a237cab113
Commit message: "[BEAM-1589] Added @OnWindowExpiration annotation. (#11350)"
 > git rev-list --no-walk 7392322bc448c16d429b0c06ca5f5965738b9f7f # timeout=10
warning: inexact rename detection was skipped due to too many files.
warning: you may want to set your diff.renameLimit variable to at least 567 and retry the command.
No emails were triggered.
[EnvInject] - Executing scripts and injecting environment variables after the SCM step.
[EnvInject] - Injecting as environment variables the properties content 

[EnvInject] - Variables injected successfully.
[Gradle] - Launching build.
[src] $ <> --continue --max-workers=12 -Dorg.gradle.jvmargs=-Xms2g -Dorg.gradle.jvmargs=-Xmx4g -Pdocker-pull-licenses :runners:twister2:validatesRunner
Starting a Gradle Daemon, 1 busy Daemon could not be reused, use --status for details
> Task :buildSrc:compileJava NO-SOURCE
> Task :buildSrc:compileGroovy FROM-CACHE
> Task :buildSrc:pluginDescriptors
> Task :buildSrc:processResources
> Task :buildSrc:classes
> Task :buildSrc:jar
> Task :buildSrc:assemble
> Task :buildSrc:spotlessGroovy
> Task :buildSrc:spotlessGroovyCheck
> Task :buildSrc:spotlessGroovyGradle
> Task :buildSrc:spotlessGroovyGradleCheck
> Task :buildSrc:spotlessCheck
> Task :buildSrc:pluginUnderTestMetadata
> Task :buildSrc:compileTestJava NO-SOURCE
> Task :buildSrc:compileTestGroovy NO-SOURCE
> Task :buildSrc:processTestResources NO-SOURCE
> Task :buildSrc:testClasses UP-TO-DATE
> Task :buildSrc:test NO-SOURCE
> Task :buildSrc:validateTaskProperties FROM-CACHE
> Task :buildSrc:check
> Task :buildSrc:build
Configuration on demand is an incubating feature.

FAILURE: Build failed with an exception.

* What went wrong:
Project 'twister2' not found in project ':runners'.

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.

* Get more help at

Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.


Publishing build scan...

Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure

To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message