pulsar-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From GitBox <...@apache.org>
Subject [GitHub] erenavsarogullari edited a comment on issue #3095: [Pulsar-Netty] Add Netty Connector Support for Tcp Messages
Date Sat, 01 Dec 2018 11:45:51 GMT
erenavsarogullari edited a comment on issue #3095: [Pulsar-Netty] Add Netty Connector Support
for Tcp Messages
URL: https://github.com/apache/pulsar/pull/3095#issuecomment-443419883
 
 
   @sijie and @srkukarni Firstly, thanks for reviews.
   
   **1-** Yep, i think about Pulsar both `Source` and `Sink` connectors for `Tcp`, `Udp` and
`Http` protocols. On the other hand, i was also thinking where these connectors can sit ideally
such as:
   - `pulsar-core` level (if the user does not use Pulsar Functions and need to work with
`Tcp`, `Udp` and `Http`. Also, he can still apply Pulsar functions on related topic) or 
   - `pulsar-io` level (if the user needs to use Pulsar functions, this connectors can be
moved into `pulsar-io` by implementing `org.apache.pulsar.io.core.PushSource` and `Sink` Interfaces).

   
   So, this is also open question for me and we can make decision.
   
   **Sample Use Cases:**
   https://netty.io/wiki/adopters.html (Netty Use Cases)
   https://keyholesoftware.com/2016/05/24/whirlpool-microservices-using-netty-and-kafka/
   http://bahir.apache.org/docs/flink/current/flink-streaming-netty/
   
   **2-** *Knative:* I think Knative support can be a required feature. As a sample high level
Pulsar pipeline on Knative:
   ```
   Source(Pulsar Event / Other Source Events(Pulsar IO Source)) -> Channel (Pulsar Channel)
--Subscription--> Service (Pulsar Functions -> Sink(e.g: Tcp/Udp/Http)))
   ```
   - Native Pulsar Support on `Knative Repo` can be useful for specially `PulsarSource` and
`PulsarChannel` (as persistent). On the other hand, `Service`(Pulsar Function)) layer can
be user-driven.
   - `Pulsar Functions / IO Sinks` can need to be supported as `runnable` and `deployable`
(through Docker in order to be managed functions' lifecycle through pods)
   
   Also, `Knative Support` can be separated `PIP`

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

Mime
View raw message