pulsar-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From zhai...@apache.org
Subject [pulsar] branch master updated: Improve documentation for message routing mode (#3263)
Date Wed, 02 Jan 2019 11:10:56 GMT
This is an automated email from the ASF dual-hosted git repository.

zhaijia pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/pulsar.git


The following commit(s) were added to refs/heads/master by this push:
     new 1c7a7a4  Improve documentation for message routing mode (#3263)
1c7a7a4 is described below

commit 1c7a7a48fe7f1ee7582184e85cec3f6bfc77161e
Author: Jia Zhai <jiazhai@users.noreply.github.com>
AuthorDate: Wed Jan 2 19:10:51 2019 +0800

    Improve documentation for message routing mode (#3263)
    
    Motivation
    User complain that the Routing modes documents is not align with code, and will cause
some mis-understanding.
    
    Modifications
    make the documentation align with the code.
---
 .../pulsar/client/api/MessageRoutingMode.java      |  8 ++++--
 site2/docs/concepts-messaging.md                   | 31 +++++++++++++++++-----
 2 files changed, 30 insertions(+), 9 deletions(-)

diff --git a/pulsar-client/src/main/java/org/apache/pulsar/client/api/MessageRoutingMode.java
b/pulsar-client/src/main/java/org/apache/pulsar/client/api/MessageRoutingMode.java
index b5cd8ba..3fb2742 100644
--- a/pulsar-client/src/main/java/org/apache/pulsar/client/api/MessageRoutingMode.java
+++ b/pulsar-client/src/main/java/org/apache/pulsar/client/api/MessageRoutingMode.java
@@ -26,12 +26,16 @@ package org.apache.pulsar.client.api;
  */
 public enum MessageRoutingMode {
     /**
-     * The producer will chose one single partition and publish all the messages into that
partition.
+     * If no key is provided, The partitioned producer will randomly pick one single partition
and publish all the messages into that partition.
+     * If a key is provided on the message, the partitioned producer will hash the key and
assign message to a particular partition.
      */
     SinglePartition,
 
     /**
-     * Publish messages across all partitions in round-robin.
+     * If no key is provided, the producer will publish messages across all partitions in
round-robin fashion to achieve maximum throughput.
+     * Please note that round-robin is not done per individual message but rather it's set
to the same boundary of batching delay, to ensure batching is effective.
+     *
+     * While if a key is specified on the message, the partitioned producer will hash the
key and assign message to a particular partition.
      */
     RoundRobinPartition,
 
diff --git a/site2/docs/concepts-messaging.md b/site2/docs/concepts-messaging.md
index c7a80a5..fae604e 100644
--- a/site2/docs/concepts-messaging.md
+++ b/site2/docs/concepts-messaging.md
@@ -199,15 +199,32 @@ Partitioned topics need to be explicitly created via the [admin API](admin-api-o
 
 When publishing to partitioned topics, you must specify a *routing mode*. The routing mode
determines which partition---that is, which internal topic---each message should be published
to.
 
-There are three routing modes available by default:
+There are three {@inject: javadoc:MessageRoutingMode:/client/org/apache/pulsar/client/api/MessageRoutingMode}
available:
 
-Mode | Description | Ordering guarantee
-:----|:------------|:------------------
-Key hash | If a key property has been specified on the message, the partitioned producer
will hash the key and assign it to a particular partition. | Per-key-bucket ordering
-Single default partition | If no key is provided, each producer's message will be routed
to a dedicated partition, initially random selected | Per-producer ordering
-Round robin distribution | If no key is provided, all messages will be routed to different
partitions in round-robin fashion to achieve maximum throughput. | None
+Mode     | Description 
+:--------|:------------
+`RoundRobinPartition` | If no key is provided, the producer will publish messages across
all partitions in round-robin fashion to achieve maximum throughput. Please note that round-robin
is not done per individual message but rather it's set to the same boundary of batching delay,
to ensure batching is effective. While if a key is specified on the message, the partitioned
producer will hash the key and assign message to a particular partition. This is the default
mode. 
+`SinglePartition`     | If no key is provided, the producer will randomly pick one single
partition and publish all the messages into that partition. While if a key is specified on
the message, the partitioned producer will hash the key and assign message to a particular
partition.
+`CustomPartition`     | Use custom message router implementation that will be called to determine
the partition for a particular message. User can create a custom routing mode by using the
[Java client](client-libraries-java.md) and implementing the {@inject: javadoc:MessageRouter:/client/org/apache/pulsar/client/api/MessageRouter}
interface.
 
-In addition to these default modes, you can also create a custom routing mode if you're using
the [Java client](client-libraries-java.md) by implementing the {@inject: javadoc:MessageRouter:/client/org/apache/pulsar/client/api/MessageRouter}
interface.
+### Ordering guarantee
+
+The ordering of messages is related to MessageRoutingMode and Message Key. Usually, user
would want an ordering of Per-key-partition guarantee.
+
+If there is a key attached to message, the messages will be routed to corresponding partitions
based on the hashing scheme specified by {@inject: javadoc:HashingScheme:/client/org/apache/pulsar/client/api/HashingScheme}
in {@inject: javadoc:ProducerBuilder:/client/org/apache/pulsar/client/api/ProducerBuilder},
when using either `SinglePartition` or `RoundRobinPartition` mode.
+
+Ordering guarantee | Description | Routing Mode and Key
+:------------------|:------------|:------------
+Per-key-partition  | All the messages with the same key will be in order and be placed in
same partition. | Use either `SinglePartition` or `RoundRobinPartition` mode, and Key is provided
by each message.
+Per-producer       | All the messages from the same producer will be in order. | Use `SinglePartition`
mode, and no Key is provided for each message.
+
+### Hashing scheme
+
+{@inject: javadoc:HashingScheme:/client/org/apache/pulsar/client/api/HashingScheme} is an
enum that represent sets of standard hashing functions available when choosing the partition
to use for a particular message.
+
+There are 2 types of standard hashing functions available: `JavaStringHash` and `Murmur3_32Hash`.

+The default hashing function for producer is `JavaStringHash`.
+Please pay attention that `JavaStringHash` is not useful when producers can be from different
multiple language clients, under this use case, it is recommended to use `Murmur3_32Hash`.
 
 
 


Mime
View raw message