metron-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From nickal...@apache.org
Subject [07/50] [abbrv] metron git commit: METRON-1701 Update General notes on the installation of Pycapa on Kerberized cluster (MohanDV via nickwallen) closes apache/metron#1136
Date Fri, 31 Aug 2018 19:20:03 GMT
METRON-1701 Update General notes on the installation of Pycapa on Kerberized cluster (MohanDV
via nickwallen) closes apache/metron#1136


Project: http://git-wip-us.apache.org/repos/asf/metron/repo
Commit: http://git-wip-us.apache.org/repos/asf/metron/commit/1b2bdff5
Tree: http://git-wip-us.apache.org/repos/asf/metron/tree/1b2bdff5
Diff: http://git-wip-us.apache.org/repos/asf/metron/diff/1b2bdff5

Branch: refs/remotes/apache/feature/METRON-1699-create-batch-profiler
Commit: 1b2bdff53106694a9223ecc97f7ce9d0a78b4b09
Parents: a1e25a4
Author: MohanDV <mohan.dv@gmail.com>
Authored: Wed Aug 1 12:13:19 2018 -0400
Committer: nickallen <nickallen@apache.org>
Committed: Wed Aug 1 12:13:19 2018 -0400

----------------------------------------------------------------------
 metron-sensors/pycapa/README.md | 47 ++++++++++++++++++++++--------------
 1 file changed, 29 insertions(+), 18 deletions(-)
----------------------------------------------------------------------


http://git-wip-us.apache.org/repos/asf/metron/blob/1b2bdff5/metron-sensors/pycapa/README.md
----------------------------------------------------------------------
diff --git a/metron-sensors/pycapa/README.md b/metron-sensors/pycapa/README.md
index 7e688a8..9f01d33 100644
--- a/metron-sensors/pycapa/README.md
+++ b/metron-sensors/pycapa/README.md
@@ -34,9 +34,9 @@ Pycapa performs network packet capture, both off-the-wire and from a Kafka
topic
 Installation
 ============
 
-General notes on the installation of Pycapa.
+General notes on the installation of Pycapa. 
 * Python 2.7 is required.
-* The following package dependencies are required and can be installed automatically with
`pip`.
+* The following package dependencies are required and can be installed automatically with
`pip`. The requirements are installed as part of step 4
   * [confluent-kafka-python](https://github.com/confluentinc/confluent-kafka-python)
   * [pcapy](https://github.com/CoreSecurity/pcapy)
 * These instructions can be used directly on CentOS 7+.  
@@ -54,8 +54,8 @@ General notes on the installation of Pycapa.
 
     ```
     export PREFIX=/usr
-    wget https://github.com/edenhill/librdkafka/archive/v0.9.4.tar.gz   -O - | tar -xz
-    cd librdkafka-0.9.4/
+    wget https://github.com/edenhill/librdkafka/archive/v0.11.5.tar.gz   -O - | tar -xz
+    cd librdkafka-0.11.5/
     ./configure --prefix=$PREFIX
     make
     make install
@@ -231,18 +231,24 @@ Capturing on 'Standard input'
 
 ### Kerberos
 
-The probe can be used in a Kerberized environment.  Follow these additional steps to use
Pycapa with Kerberos.  The following assumptions have been made.  These may need altered to
fit your environment.
+The probe can be used in a Kerberized environment. The Python client README (https://github.com/confluentinc/confluent-kafka-python)
has an important note for Kerberos case that the pre-built Linux wheels do NOT contain SASL
Kerberos support. You will need to use the non-binary wheel to install confluent-kafka-python
and build/install librdkafka separately. Follow these additional steps to use Pycapa with
Kerberos.  The following assumptions have been made.  These may need altered to fit your environment.
 
   * The Kafka broker is at `kafka1:6667`
   * Zookeeper is at `zookeeper1:2181`
   * The Kafka security protocol is `SASL_PLAINTEXT`
   * The keytab used is located at `/etc/security/keytabs/metron.headless.keytab`
   * The service principal is `metron@EXAMPLE.COM`
+   
+ 
+1. If it is not, ensure that you have `libsasl` or `libsasl2` installed.  On CentOS, this
can be installed with the following command.
+     ```
+        yum install -y cyrus-sasl cyrus-sasl-devel cyrus-sasl-gssapi
+     ```
 
 1. Build Librdkafka with SASL support (` --enable-sasl`) and install at your chosen $PREFIX.
     ```
-    wget https://github.com/edenhill/librdkafka/archive/v0.9.4.tar.gz  -O - | tar -xz
-    cd librdkafka-0.9.4/
+    wget https://github.com/edenhill/librdkafka/archive/v0.11.5.tar.gz  -O - | tar -xz
+    cd librdkafka-0.11.5/
     ./configure --prefix=$PREFIX --enable-sasl
     make
     make install
@@ -250,14 +256,19 @@ The probe can be used in a Kerberized environment.  Follow these additional
step
 
 1. Validate Librdkafka does indeed support SASL.  Run the following command and ensure that
`sasl` is returned as a built-in feature.
     ```
-    $ examples/rdkafka_example -X builtin.features
-    builtin.features = gzip,snappy,ssl,sasl,regex
-    ```
-
-   If it is not, ensure that you have `libsasl` or `libsasl2` installed.  On CentOS, this
can be installed with the following command.
-    ```
-    yum install -y cyrus-sasl cyrus-sasl-devel cyrus-sasl-gssapi
+    $ examples/rdkafka_example -X builtin.features    
+      builtin.features = gzip,snappy,ssl,sasl,regex,lz4,sasl_gssapi,sasl_plain,sasl_scram,plugins
     ```
+1. The source install of confluent-kafka.
+       
+    If you have already installed, remove the binary wheel python client first, repeat until
it says no longer installed          
+      ```       
+       pip uninstall -y confluent-kafka 
+      ```
+       
+      ```
+       pip install --no-binary :all: confluent-kafka
+      ```
 
 1. Grant access to your Kafka topic.  In this example the topic is simply named `pcap`.
     ```
@@ -279,8 +290,8 @@ The probe can be used in a Kerberized environment.  Follow these additional
step
   * `security.protocol`
   * `sasl.kerberos.keytab`
   * `sasl.kerberos.principal`
-
-        ```
+  
+    ```
         $ pycapa --producer \
             --interface eth0 \
             --kafka-broker kafka1:6667 \
@@ -292,8 +303,8 @@ The probe can be used in a Kerberized environment.  Follow these additional
step
         INFO:root:Starting packet capture
         INFO:root:Waiting for '1' message(s) to flush
         INFO:root:'10' packet(s) in, '10' packet(s) out
-        ```
-
+    ```
+    
 FAQs
 ====
 


Mime
View raw message