+

Apache Beam version 2.6.0

-
-
-Beam architecture -
-
+

We are glad to present the new 2.6.0 release of Beam. This release includes multiple fixes and new functionalities. Specifically, we spent a significant amount of time trying to automate the release, and fix the continuous integration. We also have significant new features in SQL, and portability. Please go ahead and take a look at the following highlights:

-
-
- Works with -
-
- - - - - - - - - - - - - -
-
+

For more information, please, check the detailed release notes.

-
-
-
- Testimonials -
-
- -
-
- A framework that delivers the flexibility and advanced functionality our customers need. -
-
- -
- –Talend -
-
-
- -
-
- Apache Beam has powerful semantics that solve real-world challenges of stream processing. -
-
- -
- –PayPal -
-
-
- -
-
- Apache Beam represents a principled approach for analyzing data streams. -
-
- -
- –data Artisans -
-
-
- -
-
- Beam is an open source community and contributions are greatly appreciated! - If you’d like to contribute, please see the Contribute section. -
-
-
+

New Features / Improvements

+ +

gRPC/Protobuf shading

+ +
    +
  • gRPC/protobuf is now shaded in the majority of Apache Beam Java modules. A few modules which expose gRPC/protobuf on the API surface still maintain a direct dependency.
    • +
+ +

Beam SQL

+ +
    +
  • Support for the EXISTS and LIKE operators
    • +
  • Implemented SUM() aggregations
    • +
  • Fixes to CASE expression
    • +
  • Support for date comparison
    • +
  • Support LIMIT on Unbounded Data
    • +
+ +

Portability

+ +
    +
  • Shared libraries for supporting timers and user state available for runner integration
    • +
  • A Universal Local Runner has been added to Beam. This runner runs in a single machine using portability, and containerized SDK harnesses.
    • +
  • Flink Runner now accepts jobs using the Job API
    • +
+ +

IOs

+ +
    +
  • Bounded SplittableDoFn support is now available in all runners (Note SDF is the new I/O connector API)
    • +
  • HBaseIO is the first I/O supporting Bounded SDF (using readAll)
    • +
  • Miscellaneous Fixes
    • +
+ +

SDKs

+ +
    +
  • Faster Python Avro IO
    • +
  • Python Avro IO has a use_fastavro option that will use fastavro instead of apache/avro, for a 3-6x speedup!
    • +
+ +

Other

+ +
    +
  • Updates to various dependency versions
    • +
  • Improvements on stability, performance, and documentation
    • +
+ +

List of Contributors

+

According to git shortlog, the following 39 people contributed to the 2.6.0 release. Thank you to all contributors!

+ +

Ahmet Altay, +Alan Myrvold, Alexey Romanenko, Andrew Pilloud, Ankur Goenka, Boyuan Zhang, Charles Chen, cclauss, Daniel Oliveira, Elliott Brossard, Eric Beach, Etienne Chauchot, Eugene Kirpichov, Henning Rohde, Ismaël Mejía, Kai Jiang, Kasia, Kenneth Knowles, Luis Osa, Lukasz Cwik, Maria Garcia Herrero, Mark Liu, Matthias Feys, Pablo Estrada, Rafael Fernandez, Reuven Lax, Robert Bradshaw, Robert Burke, Robin Qiu, Ryan Williams, Scott Wegner, Rui Weng, Sergei Lebedev, Sindy Li, Thomas Weise, Udi Meiri, [...] -

-
- Get started -
-
- Learn more - Download Beam SDK 2.6.0 -
-
- Java Quickstart - Python Quickstart - Go Quickstart
-
+ +