pulsar-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From GitBox <...@apache.org>
Subject [GitHub] codelipenghui opened a new issue #3056: Client send timeout happens on one broker
Date Mon, 26 Nov 2018 12:08:31 GMT
codelipenghui opened a new issue #3056: Client send timeout happens on one broker
URL: https://github.com/apache/pulsar/issues/3056
 
 
   1. 2018.11.25 11:00:00 Restart bookie node in the cluster. some topic's producer send message
timeout. After restart all brokers, most problem topic get right, but almost 3 topic still
with problem. Then found problem topics owner broker is same, so we restart the broker again.
All topics get right.
   2. 2018.11.26 18:48:30 Two topic can't write again, after check topic's owner broker, still
same broker, after restart the broker, problem topics get right.
   
   Some logs at that time:
   ```
   17:08:31.717 [pulsar-io-21-95] INFO  org.apache.pulsar.broker.service.ServerCnx - [/172.30.151.122:44388]
Subscribing on topic persistent://platform/passport/user.login-partition-0 / b.userbehavior.user.login
   17:08:31.717 [pulsar-io-21-95] INFO  org.apache.pulsar.broker.service.persistent.PersistentTopic
- [persistent://platform/passport/user.login-partition-0][b.userbehavior.user.login] Created
new subscription for 32
   17:08:31.717 [pulsar-io-21-95] INFO  org.apache.pulsar.broker.service.ServerCnx - [/172.30.151.122:44388]
Created subscription on topic persistent://platform/passport/user.login-partition-0 / b.userbehavior.user.login
   17:09:31.712 [pulsar-io-21-95] INFO  org.apache.pulsar.broker.service.persistent.PersistentDispatcherMultipleConsumers
- Removed consumer Consumer{subscription=PersistentSubscription{topic=persistent://platform/passport/user.login-partition-0,
name=b.userbehavior.user.login}, consumerId=32, consumerName=e9317, address=/172.30.151.122:44388}
with pending 117 acks
   ```
   Error log on log.bk
   ```
   18:59:05.091 [BookKeeperClientScheduler-OrderedScheduler-0-0] ERROR org.apache.bookkeeper.common.util.SafeRunnable
- Unexpected throwable caught
   java.lang.IllegalArgumentException: negative size: -1
    at com.google.common.base.Preconditions.badPositionIndex(Preconditions.java:1231) ~[com.google.guava-guava-21.0.jar:?]
    at com.google.common.base.Preconditions.checkPositionIndex(Preconditions.java:1222) ~[com.google.guava-guava-21.0.jar:?]
    at com.google.common.base.Preconditions.checkPositionIndex(Preconditions.java:1204) ~[com.google.guava-guava-21.0.jar:?]
    at com.google.common.collect.LinkedListMultimap$NodeIterator.<init>(LinkedListMultimap.java:327)
~[com.google.guava-guava-21.0.jar:?]
    at com.google.common.collect.LinkedListMultimap$1ValuesImpl.listIterator(LinkedListMultimap.java:760)
~[com.google.guava-guava-21.0.jar:?]
    at java.util.AbstractList.listIterator(AbstractList.java:299) ~[?:1.8.0_181]
    at java.util.AbstractSequentialList.iterator(AbstractSequentialList.java:239) ~[?:1.8.0_181]
    at org.apache.bookkeeper.proto.PerChannelBookieClient.checkTimeoutOnPendingOperations(PerChannelBookieClient.java:803)
~[org.apache.bookkeeper-bookkeeper-server-4.7.1.jar:4.7.1]
    at org.apache.bookkeeper.proto.DefaultPerChannelBookieClientPool.checkTimeoutOnPendingOperations(DefaultPerChannelBookieClientPool.java:100)
~[org.apache.bookkeeper-bookkeeper-server-4.7.1.jar:4.7.1]
   ```
   Client send timeout stats:
   https://files.slack.com/files-pri/T5Z3B324U-FEB7PR341/image.png

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

Mime
View raw message