beam-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From apill...@apache.org
Subject [beam] branch master updated (0e0aed9 -> 7724cfe)
Date Fri, 09 Jul 2021 18:26:51 GMT
This is an automated email from the ASF dual-hosted git repository.

apilloud pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/beam.git.


    from 0e0aed9  [BEAM-12532] correctly handle LocalDateTime and LocalTime with 0 seco…
     add 7724cfe  [BEAM-8933] Add support for reading Arrow over the BigQuery Storage API
(#14586)

No new revisions were added by this update.

Summary of changes:
 .../org/apache/beam/gradle/BeamModulePlugin.groovy |   4 +
 .../apache/beam/sdk/schemas/CachingFactory.java    |   2 +-
 .../{sql/perf-tests => arrow}/build.gradle         |  24 +-
 .../beam/sdk/extensions/arrow/ArrowConversion.java | 513 +++++++++++++++
 .../beam/sdk/extensions/arrow}/package-info.java   |   4 +-
 .../sdk/extensions/arrow/ArrowConversionTest.java  | 176 +++++
 .../BigQueryHllSketchCompatibilityIT.java          |   2 +
 sdks/java/io/google-cloud-platform/build.gradle    |   4 +
 .../beam/sdk/io/gcp/bigquery/BigQueryIO.java       |  15 +
 .../gcp/bigquery/BigQueryStorageArrowReader.java   | 107 +++
 .../io/gcp/bigquery/BigQueryStorageAvroReader.java |  80 +++
 .../gcp/bigquery/BigQueryStorageQuerySource.java   |  34 +-
 ...riteRequest.java => BigQueryStorageReader.java} |  29 +-
 ...uest.java => BigQueryStorageReaderFactory.java} |  25 +-
 .../io/gcp/bigquery/BigQueryStorageSourceBase.java |  27 +-
 .../gcp/bigquery/BigQueryStorageStreamSource.java  |  31 +-
 .../gcp/bigquery/BigQueryStorageTableSource.java   |  18 +-
 .../gcp/bigquery/BigQueryIOStorageQueryTest.java   |  22 +-
 .../io/gcp/bigquery/BigQueryIOStorageReadIT.java   |  23 +-
 .../io/gcp/bigquery/BigQueryIOStorageReadTest.java | 723 ++++++++++++++++++++-
 .../io/gcp/bigquery/BigQueryStorageReaderTest.java |  92 +++
 settings.gradle.kts                                |   1 +
 22 files changed, 1862 insertions(+), 94 deletions(-)
 copy sdks/java/extensions/{sql/perf-tests => arrow}/build.gradle (61%)
 create mode 100644 sdks/java/extensions/arrow/src/main/java/org/apache/beam/sdk/extensions/arrow/ArrowConversion.java
 copy {examples/java/src/main/java/org/apache/beam/examples/complete/kafkatopubsub/options
=> sdks/java/extensions/arrow/src/main/java/org/apache/beam/sdk/extensions/arrow}/package-info.java
(89%)
 create mode 100644 sdks/java/extensions/arrow/src/test/java/org/apache/beam/sdk/extensions/arrow/ArrowConversionTest.java
 create mode 100644 sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigquery/BigQueryStorageArrowReader.java
 create mode 100644 sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigquery/BigQueryStorageAvroReader.java
 copy sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigquery/{AvroWriteRequest.java
=> BigQueryStorageReader.java} (60%)
 copy sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigquery/{AvroWriteRequest.java
=> BigQueryStorageReaderFactory.java} (60%)
 create mode 100644 sdks/java/io/google-cloud-platform/src/test/java/org/apache/beam/sdk/io/gcp/bigquery/BigQueryStorageReaderTest.java

Mime
View raw message