beam-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From dhalperi <...@git.apache.org>
Subject [GitHub] beam pull request #2471: [BEAM-1907] PubsubIO: remove support for BoundedRea...
Date Sat, 08 Apr 2017 12:33:42 GMT
GitHub user dhalperi opened a pull request:

    https://github.com/apache/beam/pull/2471

    [BEAM-1907] PubsubIO: remove support for BoundedReader

    Google Cloud Pub/Sub is not currently that useful in bounded mode --
    it's a streaming source. Years ago, before the DirectRunner supported
    unbounded PCollections and sources, however, we were unable to run the
    streaming source in any SDK -- so we added a trivial bounded mode for
    testing.
    
    That trivial mode is no longer necessary. Additionally, it may confuse
    users into thinking it's reliable (it's not), performant (it's not),
    or has well defined semantics (it doesn't) -- it's really intended just
    for testing.
    
    Now that the DirectRunner supports everything we need -- unbounded
    PCollections, non-blocking execution with cancelation, etc. -- we can
    delete the bounded mode.

You can merge this pull request into a Git repository by running:

    $ git pull https://github.com/dhalperi/beam delete-pubsub-bounded

Alternatively you can review and apply these changes as the patch at:

    https://github.com/apache/beam/pull/2471.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #2471
    
----
commit 8f85169b8a19538fb3f59ad992c224dbc9a1e13e
Author: Dan Halperin <dhalperi@google.com>
Date:   2017-04-07T21:50:42Z

    PubsubIO: remove support for BoundedReader
    
    Google Cloud Pub/Sub is not currently that useful in bounded mode --
    it's a streaming source. Years ago, before the DirectRunner supported
    unbounded PCollections and sources, however, we were unable to run the
    streaming source in any SDK -- so we added a trivial bounded mode for
    testing.
    
    That trivial mode is no longer necessary. Additionally, it may confuse
    users into thinking it's reliable (it's not), performant (it's not),
    or has well defined semantics (it doesn't) -- it's really intended just
    for testing.
    
    Now that the DirectRunner supports everything we need -- unbounded
    PCollections, non-blocking execution with cancelation, etc. -- we can
    delete the bounded mode.

----


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

Mime
View raw message