pulsar-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From mme...@apache.org
Subject [pulsar] branch master updated: Added link to get individual connectors on download page (#3635)
Date Thu, 21 Feb 2019 06:12:52 GMT
This is an automated email from the ASF dual-hosted git repository.

mmerli pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/pulsar.git


The following commit(s) were added to refs/heads/master by this push:
     new 5e4283e  Added link to get individual connectors on download page (#3635)
5e4283e is described below

commit 5e4283e912e545cecac8c521bc32ce39f837056f
Author: Matteo Merli <mmerli@apache.org>
AuthorDate: Wed Feb 20 22:12:48 2019 -0800

    Added link to get individual connectors on download page (#3635)
---
 site2/docs/io-quickstart.md        | 35 ++++++---------
 site2/website/data/connectors.js   | 92 ++++++++++++++++++++++++++++++++++++++
 site2/website/pages/en/download.js | 70 +++++++++++++++++++++++++++++
 3 files changed, 176 insertions(+), 21 deletions(-)

diff --git a/site2/docs/io-quickstart.md b/site2/docs/io-quickstart.md
index 96328c3..39cd3d7 100644
--- a/site2/docs/io-quickstart.md
+++ b/site2/docs/io-quickstart.md
@@ -42,31 +42,20 @@ $ cd apache-pulsar-{{pulsar:version}}
 
 ## Installing Builtin Connectors
 
-Since release `2.1.0-incubating`, Pulsar releases a separate binary distribution, containing
all the `builtin` connectors.
-If you would like to enable those `builtin` connectors, you can download the connectors tarball
release in one of the following ways:
+Since release `2.3.0`, Pulsar releases all the `builtin` connectors as individual archives.
+If you would like to enable those `builtin` connectors, you can download the connectors "NAR"
+archives and from the Pulsar [downloads page](pulsar:download_page_url).
 
-* by clicking the link below and downloading the release from an Apache mirror:
-
-  * <a href="pulsar:connector_release_url" download>Pulsar IO Connectors {{pulsar:version}}
release</a>
-
-* from the Pulsar [downloads page](pulsar:download_page_url)
-* from the Pulsar [releases page](https://github.com/apache/pulsar/releases/latest)
-* using [wget](https://www.gnu.org/software/wget):
-
-  ```shell
-  $ wget pulsar:connector_release_url
-  ```
+After downloading the desired builtin connectors, these archives should be places under
+the `connectors` directory where you have unpacked the Pulsar distribution.
 
-Once the tarball is downloaded, in the pulsar directory, untar the io-connectors package
and copy the connectors as `connectors`
-in the pulsar directory:
 
 ```bash
-$ tar xvfz /path/to/apache-pulsar-io-connectors-{{pulsar:version}}-bin.tar.gz
-
-// you will find a directory named `apache-pulsar-io-connectors-{{pulsar:version}}` in the
pulsar directory
-// then copy the connectors
-
-$ cp -r apache-pulsar-io-connectors-{{pulsar:version}}/connectors connectors
+# Unpack regular Pulsar tarball and copy connectors NAR archives
+$ tar xvfz /path/to/apache-pulsar-{{pulsar:version}}-bin.tar.gz
+$ cd apache-pulsar-{{pulsar:version}}
+$ mkdir connectors
+$ cp -r /path/to/downloaded/connectors/*.nar ./connectors
 
 $ ls connectors
 pulsar-io-aerospike-{{pulsar:version}}.nar
@@ -78,6 +67,10 @@ pulsar-io-twitter-{{pulsar:version}}.nar
 ...
 ```
 
+> #### Tip
+>
+> You can also use the Docker image `apachepulsar/pulsar-all:{{pulsar:version}}` which
already
+> comes with all the available builtin connectors.
 
 ## Start Pulsar Service
 
diff --git a/site2/website/data/connectors.js b/site2/website/data/connectors.js
new file mode 100644
index 0000000..a412767
--- /dev/null
+++ b/site2/website/data/connectors.js
@@ -0,0 +1,92 @@
+module.exports = [
+    {
+        name: 'aerospike',
+        longName: 'Aerospike',
+        type: 'Sink',
+        link: 'https://www.aerospike.com/'
+    },
+    {
+        name: 'canal',
+        longName: 'Alibaba Canal CDC',
+        type: 'Source',
+        link: 'https://github.com/alibaba/canal'
+    },
+    {
+        name: 'cassandra',
+        longName: 'Apache Cassandra',
+        type: 'Sink',
+        link: 'https://cassandra.apache.org'
+    },
+    {
+        name: 'data-generator',
+        longName: 'Test Data Generator',
+        type: 'Source',
+        link: ''
+    },
+    {
+        name: 'elastic-search',
+        longName: 'Elastic Search',
+        type: 'Sink',
+        link: 'https://www.elastic.co/'
+    },
+    {
+        name: 'file',
+        longName: 'Read from local file',
+        type: 'Sink',
+        link: null
+    },
+    {
+        name: 'hdfs2',
+        longName: 'Apache HDFS v2',
+        type: 'Sink',
+        link: 'https://hadoop.apache.org/'
+    },
+    {
+        name: 'hdfs3',
+        longName: 'Apache HDFS v3',
+        type: 'Sink',
+        link: 'https://hadoop.apache.org/'
+    },
+    {
+        name: 'jdbc',
+        longName: 'JDBC',
+        type: 'Sink',
+        link: null
+    },
+    {
+        name: 'kafka',
+        longName: 'Apache Kafka',
+        type: 'Source, Sink',
+        link: 'https://kafka.apache.org/'
+    },
+    {
+        name: 'kinesis',
+        longName: 'AWS Kinesis',
+        type: 'Sink',
+        link: 'https://aws.amazon.com/kinesis/'
+    },
+    {
+        name: 'mongo',
+        longName: 'MongoDB',
+        type: 'Sink',
+        link: 'https://www.mongodb.com/'
+    },
+    {
+        name: 'netty',
+        longName: 'TCP/UDP with Netty',
+        type: 'Source',
+        link: 'https://netty.io/'
+    },
+    {
+        name: 'rabbitmq',
+        longName: 'RabbitMQ',
+        type: 'Source',
+        link: 'https://www.rabbitmq.com/'
+    },
+    {
+        name: 'twitter',
+        longName: 'Twitter Firehose',
+        type: 'Source',
+        link: 'https://developer.twitter.com/en/docs'
+    }
+]
diff --git a/site2/website/pages/en/download.js b/site2/website/pages/en/download.js
index 51a9bb8..4836319 100644
--- a/site2/website/pages/en/download.js
+++ b/site2/website/pages/en/download.js
@@ -11,15 +11,24 @@ const translate = require('../../server/translate.js').translate;
 
 const siteConfig = require(`${CWD}/siteConfig.js`);
 const releases = require(`${CWD}/releases.json`);
+const connectors = require(`${CWD}/data/connectors.js`);
 
 function getLatestArchiveMirrorUrl(version, type) {
     return `https://www.apache.org/dyn/mirrors/mirrors.cgi?action=download&filename=pulsar/pulsar-${version}/apache-pulsar-${version}-${type}.tar.gz`
 }
 
+function getLatestOffloadersMirrorUrl(version) {
+    return `https://www.apache.org/dyn/mirrors/mirrors.cgi?action=download&filename=pulsar/pulsar-${version}/apache-pulsar-offloaders-${version}-bin.tar.gz`
+}
+
 function distUrl(version, type) {
     return `https://www.apache.org/dist/pulsar/pulsar-${version}/apache-pulsar-${version}-${type}.tar.gz`
 }
 
+function distOffloadersUrl(version) {
+    return `https://www.apache.org/dist/pulsar/pulsar-${version}/apache-pulsar-offloaders-${version}-bin.tar.gz`
+}
+
 function archiveUrl(version, type) {
     if (version.includes('incubating')) {
         return `https://archive.apache.org/dist/incubator/pulsar/pulsar-${version}/apache-pulsar-${version}-${type}.tar.gz`
@@ -28,6 +37,14 @@ function archiveUrl(version, type) {
     }
 }
 
+function connectorDistUrl(name, version) {
+    return `https://www.apache.org/dist/pulsar/pulsar-${version}/connectors/pulsar-io-${name}-${version}.nar`
+}
+
+function connectorDownloadUrl(name, version) {
+    return `https://www.apache.org/dyn/mirrors/mirrors.cgi?action=download&filename=pulsar/pulsar-${version}/connectors/pulsar-io-${name}-${version}.nar`
+}
+
 class Download extends React.Component {
   render() {
     const latestVersion = releases[0];
@@ -82,9 +99,62 @@ class Download extends React.Component {
                     <a href={`${latestSrcArchiveUrl}.sha512`}>sha512</a>
                   </td>
                 </tr>
+                </tbody>
+              </table>
+
+              <h3><translate>Tiered storage offloaders</translate></h3>
+              <table className="versions" style={{width:'100%'}}>
+                <thead>
+                  <tr>
+                    <th><translate>Release</translate></th>
+                    <th><translate>Link</translate></th>
+                    <th><translate>Crypto files</translate></th>
+                  </tr>
+                </thead>
+                <tbody>
+                <tr>
+                    <th><translate>Offloaders</translate></th>
+                    <td>
+                      <a href={getLatestOffloadersMirrorUrl(latestVersion)}>apache-pulsar-offloaders-{latestVersion}-bin.tar.gz</a>
+                    </td>
+                    <td>
+                      <a href={`${distOffloadersUrl(latestVersion)}.asc`}>asc</a>,&nbsp;
+                      <a href={`${distOffloadersUrl(latestVersion)}.sha512`}>sha512</a>
+                    </td>
+                </tr>
               </tbody>
             </table>
 
+            <h3 id="connectors"><translate>Pulsar IO connectors</translate></h3>
+
+            <table className="versions" style={{width:'100%'}}>
+              <thead>
+                <tr>
+                  <th><translate>IO connector</translate></th>
+                  <th><translate>Archive</translate></th>
+                  <th><translate>Crypto files</translate></th>
+                </tr>
+              </thead>
+              <tbody>
+              {connectors.map(
+                connector =>
+                    <tr key={connector.name}>
+                      <td>
+                        <a href={`${connector.link}`} target="_blank">{connector.longName}</a>
+                      </td>
+                      <td>
+                        <a href={`${connectorDownloadUrl(connector.name, latestVersion)}`}
target="_blank">pulsar-io-{connector.name}-{latestVersion}.nar</a>
+                      </td>
+                      <td>
+                        <a href={`${connectorDistUrl(connector.name, latestVersion)}.asc`}>asc</a>,&nbsp;
+                        <a href={`${connectorDistUrl(connector.name, latestVersion)}.sha512`}>sha512</a>
+                      </td>
+                    </tr>
+              )}
+              </tbody>
+            </table>
+
+
             <h2><translate>Release Integrity</translate></h2>
             <MarkdownBlock>
               You must [verify](https://www.apache.org/info/verification.html) the integrity
of the downloaded files.


Mime
View raw message