beam-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From github-...@apache.org
Subject [beam] tag nightly-master updated (d780ebd -> 33d59b3)
Date Sat, 06 Mar 2021 02:55:06 GMT
This is an automated email from the ASF dual-hosted git repository.

github-bot pushed a change to tag nightly-master
in repository https://gitbox.apache.org/repos/asf/beam.git.


*** WARNING: tag nightly-master was modified! ***

    from d780ebd  (commit)
      to 33d59b3  (commit)
    from d780ebd  Remove pip version check to avoid an extra request and unactionable warning.
(#14129)
     add d79cd82  [BEAM-7929] Support column projection for Parquet Tables
     add 2447679  Merge pull request #14117: [BEAM-7929] Support column projection for Parquet
Tables
     add e039ca2  Merge pull request #14136: [BEAM-11648] Add conversion utilities for BigQuery
Storage API sink
     add 38fbc8c  Add a deadline of 30s to SplitReadStream
     add adf85a1  Merge pull request #13896 from [BEAM-11326] Add a deadline of 30s to SplitReadStream
     add 14e2ad4  Add topic and subscription overridden field into pubsub proto.
     add d9fd515  Merge pull request #14143 from Add topic and subscription overridden field
into pubsub proto.
     add c47ce74  Remove experiement --enable_streaming_auto_sharding
     add 4390b70  Merge pull request #14133 from [BEAM-11772, BEAM-11408] Remove Dataflow
experiment --enable_streaming_auto_sharding
     add ea21fad  [BEAM-11748] Fix PViewToIdMapper in Samza runner to not include hashcode
as part of the id (#14105)
     add 73650e3  [BEAM-11924] Add Java TextIO.read() to TestExpansionService
     add b9bb3a6  Merge pull request #14149 from ihji/BEAM-11924
     add e75e75e  Bump pandas version to 1.2.3 in python container (#14157)
     add 33d59b3  Increase the timeout  for ValidatesContainer suite (#14151)

No new revisions were added by this update.

Summary of changes:
 ...ommit_Python_ValidatesContainer_Dataflow.groovy |   2 +-
 .../pipeline/src/main/proto/beam_runner_api.proto  |  12 +
 .../beam/runners/dataflow/DataflowRunner.java      |   7 +-
 .../dataflow/DataflowPipelineTranslatorTest.java   |  46 +-
 .../beam/runners/dataflow/DataflowRunnerTest.java  |   1 -
 .../runners/samza/translation/PViewToIdMapper.java |  17 +-
 sdks/java/extensions/sql/build.gradle              |   1 +
 .../sql/meta/provider/parquet/ParquetTable.java    | 132 +++++
 .../provider/parquet/ParquetTableProvider.java     |  22 +-
 .../provider/parquet/ParquetTableProviderTest.java |  35 +-
 .../io/gcp/bigquery/BeamRowToStorageApiProto.java  | 336 +++++++++++
 .../sdk/io/gcp/bigquery/BigQueryServicesImpl.java  |  26 +-
 .../beam/sdk/io/gcp/bigquery/BigQueryUtils.java    | 221 +------
 .../beam/sdk/io/gcp/bigquery/CivilTimeEncoder.java | 648 +++++++++++++++++++++
 .../io/gcp/bigquery/TableRowToStorageApiProto.java | 304 ++++++++++
 .../gcp/bigquery/BeamRowToStorageApiProtoTest.java | 391 +++++++++++++
 .../sdk/io/gcp/bigquery/BigQueryUtilsTest.java     | 260 ---------
 .../bigquery/TableRowToStorageApiProtoTest.java    | 312 ++++++++++
 .../sdk/io/parquet/ParquetSchemaIOProvider.java    | 127 ----
 .../testing/expansion/TestExpansionService.java    |  17 +
 .../runners/dataflow/dataflow_runner_test.py       |  43 +-
 .../runners/dataflow/ptransform_overrides.py       |  15 +-
 sdks/python/container/base_image_requirements.txt  |   2 +-
 23 files changed, 2277 insertions(+), 700 deletions(-)
 create mode 100644 sdks/java/extensions/sql/src/main/java/org/apache/beam/sdk/extensions/sql/meta/provider/parquet/ParquetTable.java
 create mode 100644 sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigquery/BeamRowToStorageApiProto.java
 create mode 100644 sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigquery/CivilTimeEncoder.java
 create mode 100644 sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigquery/TableRowToStorageApiProto.java
 create mode 100644 sdks/java/io/google-cloud-platform/src/test/java/org/apache/beam/sdk/io/gcp/bigquery/BeamRowToStorageApiProtoTest.java
 create mode 100644 sdks/java/io/google-cloud-platform/src/test/java/org/apache/beam/sdk/io/gcp/bigquery/TableRowToStorageApiProtoTest.java
 delete mode 100644 sdks/java/io/parquet/src/main/java/org/apache/beam/sdk/io/parquet/ParquetSchemaIOProvider.java


Mime
View raw message