From commits-return-12193-archive-asf-public=cust-asf.ponee.io@pulsar.incubator.apache.org Wed Aug 1 22:47:13 2018 Return-Path: X-Original-To: archive-asf-public@cust-asf.ponee.io Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by mx-eu-01.ponee.io (Postfix) with SMTP id E3124180634 for ; Wed, 1 Aug 2018 22:47:12 +0200 (CEST) Received: (qmail 40506 invoked by uid 500); 1 Aug 2018 20:47:12 -0000 Mailing-List: contact commits-help@pulsar.incubator.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@pulsar.incubator.apache.org Delivered-To: mailing list commits@pulsar.incubator.apache.org Received: (qmail 40496 invoked by uid 99); 1 Aug 2018 20:47:12 -0000 Received: from ec2-52-202-80-70.compute-1.amazonaws.com (HELO gitbox.apache.org) (52.202.80.70) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 01 Aug 2018 20:47:12 +0000 From: GitBox To: commits@pulsar.apache.org Subject: [GitHub] jerrypeng commented on a change in pull request #2275: [website] improve pulsar connector documentation Message-ID: <153315643154.14458.12141552148330594597.gitbox@gitbox.apache.org> Date: Wed, 01 Aug 2018 20:47:11 -0000 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit jerrypeng commented on a change in pull request #2275: [website] improve pulsar connector documentation URL: https://github.com/apache/incubator-pulsar/pull/2275#discussion_r207025913 ########## File path: site2/docs/io-managing.md ########## @@ -0,0 +1,115 @@ +--- +id: io-managing +title: Managing Connectors +sidebar_label: Managing Connectors +--- + +This section describes how to manage Pulsar IO connectors in a Pulsar cluster. You will learn how to: + +- Deploy builtin connectors +- Monitor and update running connectors with Pulsar Admin CLI +- Deploy customized connectors +- Upgrade a connector + +## Using Builtin Connectors + +Pulsar bundles several [builtin connectors](io-overview.md#working-with-connectors) that should be used for moving data in and out +of commonly used systems such as databases, messaging systems. Getting set up to use these builtin connectors is simple. You can follow +the [instructions](getting-started-standalone.md#installing-builtin-connectors) on installing builtin connectors. After setup, all +the builtin connectors will be automatically discovered by Pulsar brokers (or function-workers), so no additional installation steps are +required. + +## Configuring Connectors + +Configuring Pulsar IO connectors is straightforward. What you need to do is to provide a yaml configuration file when your [run connectors](#running-connectors). +The yaml configuration file basically tells Pulsar where to locate the sources and sinks and how to connect those sources and sinks with Pulsar topics. + +Below is an example yaml configuration file for Cassandra Sink: + +```shell +tenant: public +namespace: default +name: cassandra-test-sink +... +# cassandra specific config +configs: + roots: "localhost:9042" + keyspace: "pulsar_test_keyspace" + columnFamily: "pulsar_test_table" + keyname: "key" + columnName: "col" +``` + +The example yaml basically tells Pulsar which Cassandra cluster to connect, what is the `keyspace` and `columnFamily` to be used in Cassandra for collecting data, +and how to map a Pulsar message into Cassandra table key and columns. + +For details, consult the documentation for [individual connectors](io-overview.md#working-with-connectors). + +## Running Connectors + +Pulsar connectors can be managed using the [`source`](reference-pulsar-admin.md#source) and [`sink`](reference-pulsar-admin.md#sink) commands of the [`pulsar-admin`](reference-pulsar-admin.md) CLI tool. + +### Running sources Review comment: For built-in sources and sinks we should probably also have an example to how to submit them. In another page you go over in detail how to submit the cassandra connector but perhaps we can also explain how to run a built-in connector without submitting a jar and just by specifying the connector type ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: users@infra.apache.org With regards, Apache Git Services