kafka-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From helowken <...@git.apache.org>
Subject [GitHub] kafka pull request #4292: Is there something wrong? (for kafka 1.0)
Date Tue, 05 Dec 2017 08:44:45 GMT
GitHub user helowken opened a pull request:

    https://github.com/apache/kafka/pull/4292

    Is there something wrong? (for kafka 1.0)

       In kafka stream source code: InternalTopologyBuilder.java
    
    ![image](https://user-images.githubusercontent.com/30587995/33597792-79425f28-d9db-11e7-81f6-15e7f81df264.png)
    
    I think it should be: **globalGroups.add(node);**

You can merge this pull request into a Git repository by running:

    $ git pull https://github.com/apache/kafka 1.0

Alternatively you can review and apply these changes as the patch at:

    https://github.com/apache/kafka/pull/4292.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #4292
    
----
commit dc907d9b7b9b0ed719c228237640d5049bb8e483
Author: Apurva Mehta <apurva@confluent.io>
Date:   2017-10-05T05:27:03Z

    KAFKA-6003; Accept appends on replicas unconditionally when local producer state doesn't
exist
    
    Without this patch, if the replica's log was somehow truncated before
    the leader's, it is possible for the replica fetcher thread to
    continuously throw an OutOfOrderSequenceException because the
    incoming sequence would be non-zero and there is no local state.
    
    This patch changes the behavior so that the replica state is updated to
    the leader's state if there was no local state for the producer at the
    time of the append.
    
    Author: Apurva Mehta <apurva@confluent.io>
    
    Reviewers: Ismael Juma <ismael@juma.me.uk>, Jason Gustafson <jason@confluent.io>
    
    Closes #4004 from apurvam/KAFKA-6003-handle-unknown-producer-on-replica
    
    (cherry picked from commit 6ea4fffdd287a0c6a02c1b6dc1006b1a7b614405)
    Signed-off-by: Jason Gustafson <jason@confluent.io>

commit 6c01d68c994d966da1cf74e7127473bda2ea3a46
Author: Rajini Sivaram <rajinisivaram@googlemail.com>
Date:   2017-10-05T16:25:34Z

    KAFKA-6012; Close request metrics only after closing request handlers
    
    Author: Rajini Sivaram <rajinisivaram@googlemail.com>
    
    Reviewers: Ismael Juma <ismael@juma.me.uk>
    
    Closes #4024 from rajinisivaram/KAFKA-6012-error-metric
    
    (cherry picked from commit e40b3a2e74133de6d60599beefb65407ca4cc7dd)
    Signed-off-by: Rajini Sivaram <rajinisivaram@googlemail.com>

commit 5e2767a26b9e4662df301df9fd57843c36e2cdc7
Author: Joel Hamill <joel@joel-hamill-confluent.local>
Date:   2017-10-05T16:30:38Z

    Rename streams tutorial and quickstart
    
    Changed these topic titles:
    - Write your own Streams Applications -> Tutorial: Write a Streams Application
    - Play with a Streams Application -> Run the Streams Demo Application
    
    Author: Joel Hamill <joel@Joel-Hamill-Confluent.local>
    Author: Joel Hamill <11722533+joel-hamill@users.noreply.github.com>
    
    Reviewers: Michael G. Noll <michael@confluent.io>, Guozhang Wang <wangguoz@gmail.com>
    
    Closes #4017 from joel-hamill/joel-hamill/streams-titles

commit 1d026269e1ab0af130b78f1efadaabbc4f5a8552
Author: Randall Hauch <rhauch@gmail.com>
Date:   2017-10-05T18:23:11Z

    KAFKA-5903: Added Connect metrics to the worker and distributed herder (KIP-196)
    
    Added metrics to the Connect worker and rebalancing metrics to the distributed herder.
    
    This is built on top of #3987, and I can rebase this PR once that is merged.
    
    Author: Randall Hauch <rhauch@gmail.com>
    
    Reviewers: Ewen Cheslack-Postava <ewen@confluent.io>
    
    Closes #4011 from rhauch/kafka-5903
    
    (cherry picked from commit a47bfbcae050659d32f777ed2f4b26dda5fbdbbd)
    Signed-off-by: Ewen Cheslack-Postava <me@ewencp.org>

commit b2bb2c6e8f6d430bd84690f25c2c1a29bc1f3864
Author: Damian Guy <damian.guy@gmail.com>
Date:   2017-10-05T19:55:55Z

    KAFKA-5989: resume consumption of tasks that have state stores but no changelogging
    
    Stores where logging is disabled where never consumed as the partitions were paused, but
never resumed.
    
    Author: Damian Guy <damian.guy@gmail.com>
    
    Reviewers: Guozhang Wang <wangguoz@gmail.com>
    
    Closes #4025 from dguy/1.0

commit 42841c9f36c86585b3b7ed7de29baee2b4f8893c
Author: Derrick Or <derrickor@gmail.com>
Date:   2017-10-05T20:31:18Z

    Update docs to reflect kafka trademark status
    
    Updated a couple places in docs with the 'registered' trademark symbol.
    
    Author: Derrick Or <derrickor@gmail.com>
    
    Reviewers: Jun Rao <junrao@gmail.com>
    
    Closes #4028 from derrickdoo/kafka-trademark-status
    
    (cherry picked from commit f29203d022f110761415c9a2ae2c149b4bc5dd21)
    Signed-off-by: Jun Rao <junrao@gmail.com>

commit b113b4bd3e3802e8e6974e6754e96bd0f1de32e2
Author: Guozhang Wang <wangguoz@gmail.com>
Date:   2017-10-06T00:02:53Z

    KAFKA-5576: RocksDB upgrade to 5.8, plus one bug fix on Bytes.wrap
    
    Author: Guozhang Wang <wangguoz@gmail.com>
    
    Reviewers: Ismael Juma <ismael@juma.me.uk>, Bill Bejeck <bill@confluent.io>,
Matthias J. Sax <matthias@confluent.io>, Damian Guy <damian.guy@gmail.com>
    
    Closes #3819 from guozhangwang/KMinor-rocksDB-573
    
    (cherry picked from commit 196bcfca0c56420793f85514d1602bde564b0651)
    Signed-off-by: Guozhang Wang <wangguoz@gmail.com>

commit 32ee1a87986916619acebea3b88609cf0a1c7dc4
Author: Konstantine Karantasis <konstantine@confluent.io>
Date:   2017-10-06T18:42:04Z

    KAFKA-5953: Register all jdbc drivers available in plugin and class paths
    
    Author: Konstantine Karantasis <konstantine@confluent.io>
    
    Reviewers: Randall Hauch <rhauch@gmail.com>, Ewen Cheslack-Postava <ewen@confluent.io>
    
    Closes #4030 from kkonstantine/KAFKA-5953-Connect-classloader-isolation-may-be-broken-for-JDBC-drivers
    
    (cherry picked from commit 974d6fec93b779cd2f9036d0ea89cf1b0917de7b)
    Signed-off-by: Ewen Cheslack-Postava <me@ewencp.org>

commit ebeee3c8dea748186d00c9c88185499c46ce3949
Author: Manikumar Reddy <manikumar.reddy@gmail.com>
Date:   2017-10-06T19:51:30Z

    KAFKA-5547; Return TOPIC_AUTHORIZATION_FAILED error if no describe access for topics
    
    Author: Manikumar Reddy <manikumar.reddy@gmail.com>
    
    Reviewers: Rajini Sivaram <rajinisivaram@googlemail.com>, Ismael Juma <ismael@juma.me.uk>,
Jason Gustafson <jason@confluent.io>
    
    Closes #3924 from omkreddy/KAFKA-5547-TOPIC-AUTHRO
    
    (cherry picked from commit 10cd98cc894b88c5d1e24fc54c66361ad9914df2)
    Signed-off-by: Jason Gustafson <jason@confluent.io>

commit aa1e4c235b882ae98a457cf9176f687951473cee
Author: Apurva Mehta <apurva@confluent.io>
Date:   2017-10-06T22:01:22Z

    KAFKA-6015; Fix NPE in RecordAccumulator after ProducerId reset
    
    It is possible for batches with sequence numbers to be in the `deque` while at the same
time the in flight batches in the `TransactionManager` are removed due to a producerId reset.
    
    In this case, when the batches in the `deque` are drained, we will get a `NullPointerException`
in the background thread due to this line:
    
    ```java
    if (first.hasSequence() && first.baseSequence() != transactionManager.nextBatchBySequence(first.topicPartition).baseSequence())
    ```
    
    Particularly, `transactionManager.nextBatchBySequence` will return null, because there
no inflight batches being tracked.
    
    In this patch, we simply allow the batches in the `deque` to be drained if there are no
in flight batches being tracked in the TransactionManager. If they succeed, well and good.
If the responses come back with an error, the batces will be ultimately failed in the producer
with an `OutOfOrderSequenceException` when the response comes back.
    
    Author: Apurva Mehta <apurva@confluent.io>
    
    Reviewers: Jason Gustafson <jason@confluent.io>
    
    Closes #4022 from apurvam/KAFKA-6015-npe-in-record-accumulator
    
    (cherry picked from commit 105ab47ed90c8a0e83c159c97a8f2294c5582657)
    Signed-off-by: Jason Gustafson <jason@confluent.io>

commit 12c77b4b08aa0eb39320915a2796444420a05392
Author: Damian Guy <damian.guy@gmail.com>
Date:   2017-10-06T22:38:23Z

    KAFKA-5651; Follow-up: add with method to Materialized
    
    Add a `with(Serde keySerde, Serde valSerde)` to `Materialized` for cases where people
don't care about the state store name.
    
    Author: Damian Guy <damian.guy@gmail.com>
    
    Reviewers: Guozhang Wang <wangguoz@gmail.com>, Ismael Juma <ismael@juma.me.uk>,
Matthias J. Sax <matthias@confluent.io>
    
    Closes #4009 from dguy/materialized
    
    (cherry picked from commit 23a014052d39521a3af471b3f95809c2164820f7)
    Signed-off-by: Guozhang Wang <wangguoz@gmail.com>

commit 963733cd0166ea47a26aaf32d876fdf3a8e086c5
Author: Guozhang Wang <wangguoz@gmail.com>
Date:   2017-10-06T22:42:18Z

    MINOR: log4j improvements on assigned tasks and store changelog reader
    
    Author: Guozhang Wang <wangguoz@gmail.com>
    
    Reviewers: Matthias J. Sax <matthias@confluent.io>, Xavier Léauté <xavier@confluent.io>,
Damian Guy <damian.guy@gmail.com>, Bill Bejeck <bill@confluent.io>
    
    Closes #4031 from guozhangwang/KMinor-assigned-task-log4j
    
    (cherry picked from commit 2427a44768f9734179c957b645df0476e7cb6d05)
    Signed-off-by: Guozhang Wang <wangguoz@gmail.com>

commit d3a603cb90fc4a0ac248569694f1f15c6e0dfbf1
Author: Ismael Juma <ismael@juma.me.uk>
Date:   2017-10-06T23:45:45Z

    KAFKA-5829; Only delete producer snapshots before the recovery point
    
    Author: Ismael Juma <ismael@juma.me.uk>
    
    Reviewers: Jason Gustafson <jason@confluent.io>
    
    Closes #4023 from ijuma/kafka-5829-avoid-reading-older-segments-on-hard-shutdown
    
    (cherry picked from commit 91517e8fbd7767ba6d7f43b517f5a26b6f870585)
    Signed-off-by: Jason Gustafson <jason@confluent.io>

commit f899c735d35f264833d41d802f0ed4010b32905d
Author: Matthias J. Sax <matthias@confluent.io>
Date:   2017-10-07T00:48:34Z

    KAFKA-5362; Follow up to Streams EOS system test
    
     - improve tests to get rid of calls to `sleep` in Python
     - fixed some flaky test conditions
     - improve debugging
    
    Author: Matthias J. Sax <matthias@confluent.io>
    
    Reviewers: Damian Guy <damian.guy@gmail.com>, Bill Bejeck <bill@confluent.io>,
Guozhang Wang <wangguoz@gmail.com>
    
    Closes #3542 from mjsax/failing-eos-system-tests
    
    (cherry picked from commit 51063441d3ed4dec8a96794f085028b4b8feb20c)
    Signed-off-by: Guozhang Wang <wangguoz@gmail.com>

commit 5298731469379abcf197b26b57f89f94785fbc64
Author: Guozhang Wang <wangguoz@gmail.com>
Date:   2017-10-07T00:57:32Z

    MINOR: KIP-161 upgrade docs change
    
    Author: Guozhang Wang <wangguoz@gmail.com>
    
    Reviewers: Damian Guy <damian.guy@gmail.com>
    
    Closes #4036 from guozhangwang/KMinor-kip-161-docs
    
    (cherry picked from commit e2e8d4a57a83693b946c3e26f9be2e41a7341ab4)
    Signed-off-by: Guozhang Wang <wangguoz@gmail.com>

commit ae4f647dd2e2ea47220ed06b61f9a58ab317c637
Author: Xin Li <xin.li@trivago.com>
Date:   2017-10-09T21:15:44Z

    KAFKA-6030; Fix Integer overflow in cleanable ratio computation
    
    Author: Xin Li <Xin.Li@trivago.com>
    
    Reviewers: Ismael Juma <ismael@juma.me.uk>, Jason Gustafson <jason@confluent.io>
    
    Closes #4043 from lisa2lisa/fix
    
    (cherry picked from commit bb27215ceac9caaac446b5fe43d3f3284624fcb4)
    Signed-off-by: Jason Gustafson <jason@confluent.io>

commit 03da215f627599240b1435469d5ac14e18f0285b
Author: bartdevylder <bartdevylder@gmail.com>
Date:   2017-10-10T00:49:52Z

    KAFKA-6026; Fix for indefinite wait in KafkaFutureImpl
    
    Author: bartdevylder <bartdevylder@gmail.com>
    Author: Bart De Vylder <bartdevylder@gmail.com>
    
    Reviewers: Colin P. Mccabe <cmccabe@confluent.io>, Ismael Juma <ismael@juma.me.uk>,
Jason Gustafson <jason@confluent.io>
    
    Closes #4044 from bartdevylder/KAFKA-6026

commit dadae0a35e4e77c195c5655bc066ccf6e155f4ed
Author: Dong Lin <lindong28@gmail.com>
Date:   2017-10-10T15:29:17Z

    KAFKA-6027; Access to log should throw KafkaStorageException after the log has been marked
offline
    
    Author: Dong Lin <lindong28@gmail.com>
    
    Reviewers: Ismael Juma <ismael@juma.me.uk>, Jun Rao <junrao@gmail.com>
    
    Closes #4045 from lindong28/KAFKA-6027
    
    (cherry picked from commit 29c46ddb91fac8c0fb3f35c1511e2afdcf67fd2e)
    Signed-off-by: Jun Rao <junrao@gmail.com>

commit 1db2fc4fd72f3d3b4b737cac5dfd9c48b518747d
Author: Matthias J. Sax <matthias@confluent.io>
Date:   2017-10-10T17:03:02Z

    MINOR: fix EOS test race condition
    
    Author: Matthias J. Sax <matthias@confluent.io>
    
    Reviewers: Damian Guy <damian.guy@gmail.com>, Bill Bejeck <bill@confluent.io>,
Guozhang Wang <wangguoz@gmail.com>
    
    Closes #4048 from mjsax/fix-eos-test-race-condition
    
    (cherry picked from commit 64930cd71361853cc46ed02232213cc7ba749e77)
    Signed-off-by: Guozhang Wang <wangguoz@gmail.com>

commit 8a175b0ee0839a1395c326a0adb1c0c2217b769e
Author: Apurva Mehta <apurva@confluent.io>
Date:   2017-10-10T17:31:52Z

    KAFKA-6016; Make the reassign partitions system test use the idempotent producer
    
    With these changes, we are ensuring that the partitions being reassigned are from non-zero
offsets. We also ensure that every message in the log has producerId and sequence number.
    
    This means that it successfully reproduces https://issues.apache.org/jira/browse/KAFKA-6003.
    
    Author: Apurva Mehta <apurva@confluent.io>
    
    Reviewers: Jason Gustafson <jason@confluent.io>
    
    Closes #4029 from apurvam/KAFKA-6016-add-idempotent-producer-to-reassign-partitions
    
    (cherry picked from commit 90b5ce3f04626ee24aebd1d06588489180a4bb05)
    Signed-off-by: Jason Gustafson <jason@confluent.io>

commit e409f847f549f4fe10c552110366c646c1e44138
Author: Jason Gustafson <jason@confluent.io>
Date:   2017-10-09T20:03:20Z

    MINOR: Factor out some common group/transactional fields in request objects
    
    Author: Jason Gustafson <jason@confluent.io>
    
    Reviewers: tedyu <yuzhihong@gmail.com>, Ismael Juma <ismael@juma.me.uk>
    
    Closes #4047 from hachikuji/factor-out-some-common-fields

commit e671c1cd36caf3b767fb9cc4af58c25918292afe
Author: Jason Gustafson <jason@confluent.io>
Date:   2017-10-10T22:01:10Z

    MINOR: Avoid some unnecessary collection copies in KafkaApis
    
    Author: Jason Gustafson <jason@confluent.io>
    
    Reviewers: Manikumar Reddy <manikumar.reddy@gmail.com>, Ismael Juma <ismael@juma.me.uk>
    
    Closes #4035 from hachikuji/KAFKA-5547-followup and squashes the following commits:
    
    f6b04ce1a [Jason Gustafson] Add a couple missed common fields
    d3473b14d [Jason Gustafson] Fix compilation errors and a few warnings
    58a0ae695 [Jason Gustafson] MINOR: Avoid some unnecessary collection copies in KafkaApis
    
    (cherry picked from commit 1027ff3c769906b7b80582217cf5b4703fd6864d)
    Signed-off-by: Jason Gustafson <jason@confluent.io>

commit 85f6f8cbb353a7c6ec8606b088b1f35dfcf1f6ea
Author: Bill Bejeck <bill@confluent.io>
Date:   2017-10-10T22:07:53Z

    KAFKA-6025: small fix for streams tutorial
    
    Author: Bill Bejeck <bill@confluent.io>
    
    Reviewers: Guozhang Wang <wangguoz@gmail.com>
    
    Closes #4053 from bbejeck/KAFKA_6025_fix_streams_tutorial
    
    (cherry picked from commit dac990aab31ab13ed08a6818e1516f4dd25331dd)
    Signed-off-by: Guozhang Wang <wangguoz@gmail.com>

commit 6c1c5efe7f83afc733608122c270642ccdc97399
Author: Matthias J. Sax <matthias@confluent.io>
Date:   2017-10-10T22:13:19Z

    MINOR: KIP-182 follow up; add deprecation annotations to test classes
    
    Author: Matthias J. Sax <matthias@confluent.io>
    
    Reviewers: Bill Bejeck <bill@confluent.io>, Guozhang Wang <wangguoz@gmail.com>
    
    Closes #4051 from mjsax/minor-kip-182-follow-up
    
    (cherry picked from commit a5a9a901ef78373cda4b743323ef9f6e6da0dbf6)
    Signed-off-by: Guozhang Wang <wangguoz@gmail.com>

commit 2cf059f33d8dc64e9ee14769c60e7d6ff559ae55
Author: Guozhang Wang <wangguoz@gmail.com>
Date:   2017-10-11T21:20:41Z

    HOTFIX: Updates on release.py before 1.0.0
    
    Author: Guozhang Wang <wangguoz@gmail.com>
    
    Reviewers: Ismael Juma <ismael@juma.me.uk>, Ewen Cheslack-Postava <ewen@confluent.io>
    
    Closes #4054 from guozhangwang/KMinor-pre-1.0-release
    
    (cherry picked from commit 54ed3435bf0660295b0f0fee0df4c9761873225c)
    Signed-off-by: Guozhang Wang <wangguoz@gmail.com>

commit 531dfd7268a5642f97e2bf79453b43ba307b9c0f
Author: Ewen Cheslack-Postava <me@ewencp.org>
Date:   2017-10-11T21:32:40Z

    MINOR: Push JMX metric name mangling into the JmxReporter (KIP-190 follow up)
    
    Author: Ewen Cheslack-Postava <me@ewencp.org>
    
    Reviewers: Rajini Sivaram <rajinisivaram@googlemail.com>, Ismael Juma <ismael@juma.me.uk>
    
    Closes #3980 from ewencp/dont-mangle-names
    
    (cherry picked from commit 7d6ca52a2751908c7fc6b752d70dfaaaaa9bbe8c)
    Signed-off-by: Rajini Sivaram <rajinisivaram@googlemail.com>

commit 334f8a48baba60a973880f73c6d9e98fb0785d52
Author: Manjula K <manjula@kafka-summit.org>
Date:   2017-10-11T22:16:12Z

    MINOR: Redesign of Streams page to include video & logos
    
    guozhangwang Please review.
    
    Author: Manjula K <manjula@kafka-summit.org>
    
    Reviewers: Guozhang Wang <wangguoz@gmail.com>
    
    Closes #4059 from manjuapu/redesign-streams-page
    
    (cherry picked from commit f6e724b127891bc950499fd7a6876e500ba78de3)
    Signed-off-by: Guozhang Wang <wangguoz@gmail.com>

commit d5614b529274d7781356305fc5bb14bf80683727
Author: Apurva Mehta <apurva@confluent.io>
Date:   2017-10-12T09:05:14Z

    KAFKA-6053; Fix NoSuchMethodError when creating ProducerRecords with older client versions
    
    Author: Apurva Mehta <apurva@confluent.io>
    
    Reviewers: Ismael Juma <ismael@juma.me.uk>
    
    Closes #4057 from apurvam/KAFKA-6053-fix-no-such-method-error-in-producer-record

commit a04d18ffb37b32d0a2ffe395ae2a8b279dcf89d1
Author: Vahid Hashemian <vahidhashemian@us.ibm.com>
Date:   2017-10-12T09:09:16Z

    KAFKA-6055; Fix typo in KAFKA_JVM_PERFORMANCE_OPTS for Windows
    
    Author: Vahid Hashemian <vahidhashemian@us.ibm.com>
    
    Reviewers: Ismael Juma <ismael@juma.me.uk>
    
    Closes #4062 from vahidhashemian/KAFKA-6055
    
    (cherry picked from commit bfa82fc0b63a5ad0121da6e2190a587eb672a1d5)
    Signed-off-by: Ismael Juma <ismael@juma.me.uk>

commit c49c830690a09ab3925416df5b6429d96b034002
Author: Mickael Maison <mickael.maison@gmail.com>
Date:   2017-10-12T09:12:46Z

    MINOR: Fix log message in AbstractFetcherThread
    
    Use Scala string templates instead of format
    
    Author: Mickael Maison <mickael.maison@gmail.com>
    
    Reviewers: Ismael Juma <ismael@juma.me.uk>
    
    Closes #4058 from mimaison/minor_AFT_logging

----


---

Mime
View raw message