spark-reviews mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From koeninger <>
Subject [GitHub] spark pull request: [SPARK-4964] [Streaming] Exactly-once semantic...
Date Tue, 03 Feb 2015 17:12:02 GMT
Github user koeninger commented on the pull request:
    Regarding naming, I agree.  The name has been a point of discussion for a
    month, how to get some consensus?
    Regarding Java wrappers, there have already been a number of changes
    directed at java compatibility (array of Leader instead of a map[topic,
    broker], .create in addition to .apply).  I wonder how relevant those are
    if we're doing separate java wrappers (which yes, I agree should be in a
    follow-on PR)
    On Tue, Feb 3, 2015 at 3:13 AM, Patrick Wendell <>
    > I took a pass through the public API. I'm not very familiar with Kafka so
    > it was somewhat slow going. However, some reactions:
    >    1. We should try to tighten, simplify, and clarify the way we name and
    >    document everything in this public API. Most of the comments were about
    >    this. The most important IMO is coming up with a good name for the new
    >    streams returned and clearly explaining how they differ from the old Kafka
    >    stream. To me, the main differences seems to be in the way we (a) decide
    >    what goes into which batch and (b) actually ingest the data. I proposed
    >    javadoc and naming scheme that emphasizing that distinction.
    >    2. Is there plans to add a Java and Python wrappers here next? Those
    >    are straightforward and it would be good to have them. Maybe in a follow on
    >    PR?
    > —
    > Reply to this email directly or view it on GitHub
    > <>.

If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at or file a JIRA ticket
with INFRA.

To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message