pulsar-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From GitBox <...@apache.org>
Subject [GitHub] jerrypeng commented on a change in pull request #2275: [website] improve pulsar connector documentation
Date Wed, 01 Aug 2018 20:47:11 GMT
jerrypeng commented on a change in pull request #2275:  [website] improve pulsar connector
URL: https://github.com/apache/incubator-pulsar/pull/2275#discussion_r207025913

 File path: site2/docs/io-managing.md
 @@ -0,0 +1,115 @@
+id: io-managing
+title: Managing Connectors
+sidebar_label: Managing Connectors
+This section describes how to manage Pulsar IO connectors in a Pulsar cluster. You will learn
how to:
+- Deploy builtin connectors
+- Monitor and update running connectors with Pulsar Admin CLI
+- Deploy customized connectors
+- Upgrade a connector
+## Using Builtin Connectors
+Pulsar bundles several [builtin connectors](io-overview.md#working-with-connectors) that
should be used for moving data in and out
+of commonly used systems such as databases, messaging systems. Getting set up to use these
builtin connectors is simple. You can follow
+the [instructions](getting-started-standalone.md#installing-builtin-connectors) on installing
builtin connectors. After setup, all
+the builtin connectors will be automatically discovered by Pulsar brokers (or function-workers),
so no additional installation steps are
+## Configuring Connectors
+Configuring Pulsar IO connectors is straightforward. What you need to do is to provide a
yaml configuration file when your [run connectors](#running-connectors).
+The yaml configuration file basically tells Pulsar where to locate the sources and sinks
and how to connect those sources and sinks with Pulsar topics.
+Below is an example yaml configuration file for Cassandra Sink:
+tenant: public
+namespace: default
+name: cassandra-test-sink
+# cassandra specific config
+    roots: "localhost:9042"
+    keyspace: "pulsar_test_keyspace"
+    columnFamily: "pulsar_test_table"
+    keyname: "key"
+    columnName: "col"
+The example yaml basically tells Pulsar which Cassandra cluster to connect, what is the `keyspace`
and `columnFamily` to be used in Cassandra for collecting data,
+and how to map a Pulsar message into Cassandra table key and columns.
+For details, consult the documentation for [individual connectors](io-overview.md#working-with-connectors).
+## Running Connectors
+Pulsar connectors can be managed using the [`source`](reference-pulsar-admin.md#source) and
[`sink`](reference-pulsar-admin.md#sink) commands of the [`pulsar-admin`](reference-pulsar-admin.md)
CLI tool.
+### Running sources
 Review comment:
   For built-in sources and sinks we should probably also have an example to how to submit
them.  In another page you go over in detail how to submit the cassandra connector but perhaps
we can also explain how to run a built-in connector without submitting a jar and just by specifying
the connector type

This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:

With regards,
Apache Git Services

View raw message