beam-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From mshields822 <>
Subject [GitHub] incubator-beam pull request: [BEAM-53] Wire PubsubUnbounded{Source...
Date Tue, 17 May 2016 23:48:06 GMT
GitHub user mshields822 opened a pull request:

    [BEAM-53] Wire PubsubUnbounded{Source,Sink} into PubsubIO

    This also refines the handling of record ids in the sink to be random-but-reused-on-failure,
using the same trick as we do for the BigQuery sink.
    Still need to re-do the load tests I did a few weeks back with the actual change.
    Note that last time I tested the DataflowPipelineTranslator does not kick in and replace
the new transforms with the correct native transforms. Need to dig deeper.
    R: @dhalperi 

You can merge this pull request into a Git repository by running:

    $ git pull pubsub-runner

Alternatively you can review and apply these changes as the patch at:

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #346
commit ce655a6a7480147f7527fa23818e1d546abaa599
Author: Mark Shields <>
Date:   2016-04-12T00:36:27Z

    Make java unbounded pub/sub source the default.

commit aafcf5f9d0286ec5a6ed0d634df8ff0902897cdc
Author: Mark Shields <>
Date:   2016-05-17T23:44:30Z

    Refine record id calculation. Prepare for supporting unit tests with re-using record ids.


If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at or file a JIRA ticket
with INFRA.

View raw message