pulsar-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From GitBox <...@apache.org>
Subject [GitHub] jiazhai closed pull request #3263: Improve documentation for message routing mode
Date Wed, 02 Jan 2019 11:10:53 GMT
jiazhai closed pull request #3263: Improve documentation for message routing mode
URL: https://github.com/apache/pulsar/pull/3263
 
 
   

This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:

As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):

diff --git a/pulsar-client/src/main/java/org/apache/pulsar/client/api/MessageRoutingMode.java
b/pulsar-client/src/main/java/org/apache/pulsar/client/api/MessageRoutingMode.java
index b5cd8ba1d7..3fb27428d6 100644
--- a/pulsar-client/src/main/java/org/apache/pulsar/client/api/MessageRoutingMode.java
+++ b/pulsar-client/src/main/java/org/apache/pulsar/client/api/MessageRoutingMode.java
@@ -26,12 +26,16 @@
  */
 public enum MessageRoutingMode {
     /**
-     * The producer will chose one single partition and publish all the messages into that
partition.
+     * If no key is provided, The partitioned producer will randomly pick one single partition
and publish all the messages into that partition.
+     * If a key is provided on the message, the partitioned producer will hash the key and
assign message to a particular partition.
      */
     SinglePartition,
 
     /**
-     * Publish messages across all partitions in round-robin.
+     * If no key is provided, the producer will publish messages across all partitions in
round-robin fashion to achieve maximum throughput.
+     * Please note that round-robin is not done per individual message but rather it's set
to the same boundary of batching delay, to ensure batching is effective.
+     *
+     * While if a key is specified on the message, the partitioned producer will hash the
key and assign message to a particular partition.
      */
     RoundRobinPartition,
 
diff --git a/site2/docs/concepts-messaging.md b/site2/docs/concepts-messaging.md
index c7a80a5094..fae604ec06 100644
--- a/site2/docs/concepts-messaging.md
+++ b/site2/docs/concepts-messaging.md
@@ -199,15 +199,32 @@ Partitioned topics need to be explicitly created via the [admin API](admin-api-o
 
 When publishing to partitioned topics, you must specify a *routing mode*. The routing mode
determines which partition---that is, which internal topic---each message should be published
to.
 
-There are three routing modes available by default:
+There are three {@inject: javadoc:MessageRoutingMode:/client/org/apache/pulsar/client/api/MessageRoutingMode}
available:
 
-Mode | Description | Ordering guarantee
-:----|:------------|:------------------
-Key hash | If a key property has been specified on the message, the partitioned producer
will hash the key and assign it to a particular partition. | Per-key-bucket ordering
-Single default partition | If no key is provided, each producer's message will be routed
to a dedicated partition, initially random selected | Per-producer ordering
-Round robin distribution | If no key is provided, all messages will be routed to different
partitions in round-robin fashion to achieve maximum throughput. | None
+Mode     | Description 
+:--------|:------------
+`RoundRobinPartition` | If no key is provided, the producer will publish messages across
all partitions in round-robin fashion to achieve maximum throughput. Please note that round-robin
is not done per individual message but rather it's set to the same boundary of batching delay,
to ensure batching is effective. While if a key is specified on the message, the partitioned
producer will hash the key and assign message to a particular partition. This is the default
mode. 
+`SinglePartition`     | If no key is provided, the producer will randomly pick one single
partition and publish all the messages into that partition. While if a key is specified on
the message, the partitioned producer will hash the key and assign message to a particular
partition.
+`CustomPartition`     | Use custom message router implementation that will be called to determine
the partition for a particular message. User can create a custom routing mode by using the
[Java client](client-libraries-java.md) and implementing the {@inject: javadoc:MessageRouter:/client/org/apache/pulsar/client/api/MessageRouter}
interface.
 
-In addition to these default modes, you can also create a custom routing mode if you're using
the [Java client](client-libraries-java.md) by implementing the {@inject: javadoc:MessageRouter:/client/org/apache/pulsar/client/api/MessageRouter}
interface.
+### Ordering guarantee
+
+The ordering of messages is related to MessageRoutingMode and Message Key. Usually, user
would want an ordering of Per-key-partition guarantee.
+
+If there is a key attached to message, the messages will be routed to corresponding partitions
based on the hashing scheme specified by {@inject: javadoc:HashingScheme:/client/org/apache/pulsar/client/api/HashingScheme}
in {@inject: javadoc:ProducerBuilder:/client/org/apache/pulsar/client/api/ProducerBuilder},
when using either `SinglePartition` or `RoundRobinPartition` mode.
+
+Ordering guarantee | Description | Routing Mode and Key
+:------------------|:------------|:------------
+Per-key-partition  | All the messages with the same key will be in order and be placed in
same partition. | Use either `SinglePartition` or `RoundRobinPartition` mode, and Key is provided
by each message.
+Per-producer       | All the messages from the same producer will be in order. | Use `SinglePartition`
mode, and no Key is provided for each message.
+
+### Hashing scheme
+
+{@inject: javadoc:HashingScheme:/client/org/apache/pulsar/client/api/HashingScheme} is an
enum that represent sets of standard hashing functions available when choosing the partition
to use for a particular message.
+
+There are 2 types of standard hashing functions available: `JavaStringHash` and `Murmur3_32Hash`.

+The default hashing function for producer is `JavaStringHash`.
+Please pay attention that `JavaStringHash` is not useful when producers can be from different
multiple language clients, under this use case, it is recommended to use `Murmur3_32Hash`.
 
 
 


 

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

Mime
View raw message