beam-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From reuvenlax <>
Subject [GitHub] beam pull request #1952: BEAM-1438 Auto shard streaming sinks
Date Wed, 08 Feb 2017 21:09:49 GMT
GitHub user reuvenlax opened a pull request:

    BEAM-1438 Auto shard streaming sinks

    If a Write requests runner-determined sharding, per-bundle sharding is the default but
performs poorly in Dataflow's streaming runner. Instead, the runner statically picks a sharding
based on the number of workers.

You can merge this pull request into a Git repository by running:

    $ git pull streaming_auto_shard_write

Alternatively you can review and apply these changes as the patch at:

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #1952
commit a5894fd416b07b0ede094022ccf6539767507c7f
Author: Reuven Lax <>
Date:   2016-11-14T04:29:31Z

    Add PubSub attributes support to PubsubIO.

commit 459dbd2089766f4ab7b631ed1819d938da7ba5c8
Author: Reuven Lax <>
Date:   2017-02-08T20:53:27Z

    If a Write operation requests runner-determined sharding, make the Dataflow runner default
to maxNumWorkers * 2 shards.

commit 1d3a5239e3a6bd5646638e90578d8a8034fa78d1
Author: Reuven Lax <>
Date:   2017-02-08T20:57:33Z

    Fix nullable attributes


If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at or file a JIRA ticket
with INFRA.

View raw message