eagle-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From jjtang <r7raul1...@163.com>
Subject Re:Re: Instructions on installing Eagle 0.5 on HDP 2.5
Date Sat, 18 Mar 2017 07:15:56 GMT

Hi Markovich,
Apache Eagle 0.5 is well supported with storm 0.9.3 and kafka 0.8.x .




At 2017-03-18 10:53:37, "Jayesh Senjaliya" <jaysen@apache.org> wrote:

Hi Markovich,


Eagle 0.5 is well supported with storm 0.9.3 , can you please try using that ?


or do you have to use it with storm 1.x ?


- Jayesh




On Fri, Mar 17, 2017 at 6:50 AM, Markovich <amrivkin@gmail.com> wrote:

Hello eagle users and dev,


I'm stuck on installing eagle on HDP2.5 cluster with jdk 1.8.0_101.


Here is my service versions and my cluster is secured using kerberos + ranger. 


HDFS2.7.3

Hive1.2.1000

Storm1.0.1

Kafka0.10.0

Kerberos1.10.3-10



Here is what I've done already:


1) Downloaded latest eagle from github (version 0.5 snapshot).
2) Builded it using mvn clean package -DskipTests 
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 16:11 min
[INFO] Finished at: 2017-03-17T16:00:45+03:00
[INFO] Final Memory: 183M/1755M
[INFO] ------------------------------------------------------------------------


3) Moved tarball to /usr/hdp/current/ and extracted to eagle
4) Changed conf/eagle.conf for my cluster:
zkQuorum

zookeeperZnodeParent = "/hbase-secure"

metadata -> jdbc -> user, pass and host
nimbusHost

5) Launched bin/eagle-env.sh and bin/eagle-server start


Eagle started on 9090 port. 
6) In Web UI entered SiteId
7) Selected install on HDFS Audit Log Monitor and chenged General settings (Kafka hosts) and
Advanced fs.defaultFS. Execution Mode - cluster
8) Created hdfs_audit_log_{SITE_ID}, hdfs_audit_log_enriched_{SITE_ID}
9) Launched Logstash to write to hdfs_audit_log_{SITE_ID}
10) Checked, logs are pushed in kafka
11) Appliaction is initialized
12) Tried to start this appliaction:


INFO  [2017-03-17 13:28:45,926] org.apache.eagle.dataproc.impl.storm.kafka.KafkaSpoutProvider:
Use topic : hdfs_audit_log, zkConnection : localhost:2181 , fetchSize : 1048576
WARN  [2017-03-17 13:28:46,216] org.apache.eagle.app.messaging.KafkaStreamProvider: Using
default shared sink topic dataSinkConfig.topic: hdfs_audit_event
INFO  [2017-03-17 13:28:46,331] org.apache.eagle.app.environment.impl.StormExecutionRuntime:
Starting HDFS_AUDIT_LOG_MONITOR_APP_MYSITE
 (org.apache.eagle.security.auditlog.HdfsAuditLogApplication), mode: CLUSTER
INFO  [2017-03-17 13:28:46,332] org.apache.eagle.app.environment.impl.StormExecutionRuntime:
Overriding application.storm.nimbusHost = demo5
INFO  [2017-03-17 13:28:46,332] org.apache.eagle.app.environment.impl.StormExecutionRuntime:
Overriding application.storm.nimbusThriftPort = 6627
INFO  [2017-03-17 13:28:46,332] org.apache.eagle.app.environment.impl.StormExecutionRuntime:
Submitting as cluster mode ...


Nothing in Storm UI. Nothing in logs.
Tried to change KafkaSpoutProvider to normal with SITE_ID. -> Nothing changed. 


I think I need to enter some kerberos related configs for storm...


13) Ok, changed  Execution Mode - Local. 
14) Started. Status in UI changed to starting. In logs a lot of activity, but also errors:


WARN  [2017-03-17 13:33:40,952] storm.kafka.KafkaUtils: there are more tasks than partitions
(tasks: 2; partitions: 1), some tasks will be idle
INFO  [2017-03-17 13:33:40,952] storm.kafka.KafkaUtils: Task [1/2] assigned [Partition{host=null:-1,
partition=0}]
INFO  [2017-03-17 13:33:40,952] storm.kafka.ZkCoordinator: Task [1/2] Deleted partition managers:
[]
INFO  [2017-03-17 13:33:40,952] storm.kafka.ZkCoordinator: Task [1/2] New partition managers:
[Partition{host=null:-1, partition=0}]
INFO  [2017-03-17 13:33:40,990] storm.kafka.PartitionManager: Read partition information from:
/consumers/hdfs_audit_log_tuskpro/eagleConsumer/partition_0  --> null
ERROR [2017-03-17 13:33:41,047] backtype.storm.util: Async loop died!
! java.lang.NullPointerException: null
! at org.apache.kafka.common.utils.Utils.formatAddress(Utils.java:312) ~[eagle-topology-0.5.0-SNAPSHOT-assembly.jar:na]
! at kafka.consumer.SimpleConsumer$$anonfun$disconnect$1.apply(SimpleConsumer.scala:49) ~[eagle-topology-0.5.0-SNAPSHOT-assembly.jar:na]
! at kafka.consumer.SimpleConsumer$$anonfun$disconnect$1.apply(SimpleConsumer.scala:49) ~[eagle-topology-0.5.0-SNAPSHOT-assembly.jar:na]
! at kafka.utils.Logging$class.debug(Logging.scala:52) ~[eagle-topology-0.5.0-SNAPSHOT-assembly.jar:na]
! at kafka.consumer.SimpleConsumer.debug(SimpleConsumer.scala:30) ~[eagle-topology-0.5.0-SNAPSHOT-assembly.jar:na]
! at kafka.consumer.SimpleConsumer.disconnect(SimpleConsumer.scala:49) ~[eagle-topology-0.5.0-SNAPSHOT-assembly.jar:na]
! at kafka.consumer.SimpleConsumer.liftedTree1$1(SimpleConsumer.scala:82) ~[eagle-topology-0.5.0-SNAPSHOT-assembly.jar:na]
! at kafka.consumer.SimpleConsumer.kafka$consumer$SimpleConsumer$$sendRequest(SimpleConsumer.scala:68)
~[eagle-topology-0.5.0-SNAPSHOT-assembly.jar:na]
! at kafka.consumer.SimpleConsumer.getOffsetsBefore(SimpleConsumer.scala:127) ~[eagle-topology-0.5.0-SNAPSHOT-assembly.jar:na]
! at kafka.javaapi.consumer.SimpleConsumer.getOffsetsBefore(SimpleConsumer.scala:79) ~[eagle-topology-0.5.0-SNAPSHOT-assembly.jar:na]
! at storm.kafka.KafkaUtils.getOffset(KafkaUtils.java:74) ~[eagle-topology-0.5.0-SNAPSHOT-assembly.jar:na]
! at storm.kafka.KafkaUtils.getOffset(KafkaUtils.java:64) ~[eagle-topology-0.5.0-SNAPSHOT-assembly.jar:na]
! at storm.kafka.PartitionManager.<init>(PartitionManager.java:89) ~[eagle-topology-0.5.0-SNAPSHOT-assembly.jar:na]
! at storm.kafka.ZkCoordinator.refresh(ZkCoordinator.java:98) ~[eagle-topology-0.5.0-SNAPSHOT-assembly.jar:na]
! ... 6 common frames omitted
! Causing: java.lang.RuntimeException: java.lang.NullPointerException
! at storm.kafka.ZkCoordinator.refresh(ZkCoordinator.java:103) ~[eagle-topology-0.5.0-SNAPSHOT-assembly.jar:na]
! at storm.kafka.ZkCoordinator.getMyManagedPartitions(ZkCoordinator.java:69) ~[eagle-topology-0.5.0-SNAPSHOT-assembly.jar:na]
! at storm.kafka.KafkaSpout.nextTuple(KafkaSpout.java:135) ~[eagle-topology-0.5.0-SNAPSHOT-assembly.jar:na]
! at backtype.storm.daemon.executor$fn__3373$fn__3388$fn__3417.invoke(executor.clj:565) ~[storm-core-0.9.3.jar:0.9.3]
! at backtype.storm.util$async_loop$fn__464.invoke(util.clj:463) ~[storm-core-0.9.3.jar:0.9.3]
! at clojure.lang.AFn.run(AFn.java:24) [eagle-topology-0.5.0-SNAPSHOT-assembly.jar:na]
! at java.lang.Thread.run(Thread.java:745) [na:1.8.0_101]


....


ERROR [2017-03-17 13:33:41,079] backtype.storm.util: Halting process: ("Worker died")
! java.lang.RuntimeException: ("Worker died")
! at backtype.storm.util$exit_process_BANG_.doInvoke(util.clj:325) [storm-core-0.9.3.jar:0.9.3]
! at clojure.lang.RestFn.invoke(RestFn.java:423) [eagle-topology-0.5.0-SNAPSHOT-assembly.jar:na]
! at backtype.storm.daemon.worker$fn__3808$fn__3809.invoke(worker.clj:452) [storm-core-0.9.3.jar:0.9.3]
! at backtype.storm.daemon.executor$mk_executor_data$fn__3274$fn__3275.invoke(executor.clj:240)
[storm-core-0.9.3.jar:0.9.3]
! at backtype.storm.util$async_loop$fn__464.invoke(util.clj:473) [storm-core-0.9.3.jar:0.9.3]
! at clojure.lang.AFn.run(AFn.java:24) [eagle-topology-0.5.0-SNAPSHOT-assembly.jar:na]
! at java.lang.Thread.run(Thread.java:745) [na:1.8.0_101]


ERROR [2017-03-17 13:34:10,013] org.apache.eagle.security.enrich.DataEnrichJob: Fail to load
sensitivity data
! java.net.ConnectException: Connection refused
! at java.net.PlainSocketImpl.socketConnect(Native Method) ~[na:1.8.0_101]
! at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350) ~[na:1.8.0_101]
! at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206) ~[na:1.8.0_101]
! at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188) ~[na:1.8.0_101]
! at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) ~[na:1.8.0_101]
! at java.net.Socket.connect(Socket.java:589) ~[na:1.8.0_101]
! at sun.net.NetworkClient.doConnect(NetworkClient.java:175) ~[na:1.8.0_101]
! at sun.net.www.http.HttpClient.openServer(HttpClient.java:432) ~[na:1.8.0_101]
! at sun.net.www.http.HttpClient.openServer(HttpClient.java:527) ~[na:1.8.0_101]
! at sun.net.www.http.HttpClient.<init>(HttpClient.java:211) ~[na:1.8.0_101]
! at sun.net.www.http.HttpClient.New(HttpClient.java:308) ~[na:1.8.0_101]
! at sun.net.www.http.HttpClient.New(HttpClient.java:326) ~[na:1.8.0_101]
! at sun.net.www.protocol.http.HttpURLConnection.getNewHttpClient(HttpURLConnection.java:1169)
~[na:1.8.0_101]
! at sun.net.www.protocol.http.HttpURLConnection.plainConnect0(HttpURLConnection.java:1105)
~[na:1.8.0_101]
! at sun.net.www.protocol.http.HttpURLConnection.plainConnect(HttpURLConnection.java:999)
~[na:1.8.0_101]
! at sun.net.www.protocol.http.HttpURLConnection.connect(HttpURLConnection.java:933) ~[na:1.8.0_101]
! at sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1513)
~[na:1.8.0_101]
! at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1441)
~[na:1.8.0_101]
! at java.net.HttpURLConnection.getResponseCode(HttpURLConnection.java:480) ~[na:1.8.0_101]
! at com.sun.jersey.client.urlconnection.URLConnectionClientHandler._invoke(URLConnectionClientHandler.java:253)
~[jersey-client-1.19.1.jar:1.19.1]
! at com.sun.jersey.client.urlconnection.URLConnectionClientHandler.handle(URLConnectionClientHandler.java:153)
~[jersey-client-1.19.1.jar:1.19.1]
! ... 12 common frames omitted
! Causing: com.sun.jersey.api.client.ClientHandlerException: java.net.ConnectException: Connection
refused
! at com.sun.jersey.client.urlconnection.URLConnectionClientHandler.handle(URLConnectionClientHandler.java:155)
~[jersey-client-1.19.1.jar:1.19.1]
! at com.sun.jersey.api.client.filter.GZIPContentEncodingFilter.handle(GZIPContentEncodingFilter.java:123)
~[jersey-client-1.19.1.jar:1.19.1]
! at com.sun.jersey.api.client.Client.handle(Client.java:652) ~[jersey-client-1.19.1.jar:1.19.1]


Ui wasn't accessable, but server was still running. 
It was failing again and again.


13) Restarted server
14) Nothing was saved. Again fresh install. UI asks for site_id.
15) Checked Hbase and MySQL Server - all is empty.




So can someone please help me to get started with Eagle on my cluster?




Also here is Logstash info:
logstash-5.2.2





  output {                                                                               
                 
                                                                                         
                 
      if [type] == "hdp-nn-audit" {                                                      
                 
        kafka   {                                                                        
                 
                codec => plain {format => "%{message}"}                            
                       
                bootstrap_servers => "demo4:6667"                                     
             
                topic_id => "hdfs_audit_log_tuskpro"                                  
                    
                security_protocol => "SASL_PLAINTEXT"                                 
                    
                sasl_kerberos_service_name => "kafka"                                 
                    
                jaas_path => "/usr/hdp/current/kafka-broker/config/kafka_client_jaas.conf"
                
                kerberos_config => "/etc/krb5.conf"                                   
                    
                client_id => "hdp-nn-audit"                                           
                    
                message_key => "%{user}"                                              
                    
                }                                                                        
                 
                                  }                                                      
                 
         }       




Also I don't know why, but logstash is not using kafka_client_jaas. 
There is info about sasl_kerberos_service_name , but without this property logstash is not
working. 


ยท
KafkaClient {                                                                            
                                                                               
   com.sun.security.auth.module.Krb5LoginModule required                                 
                                                                               
   useTicketCache=true                                                                   
                                                                               
   renewTicket=true                                                                      
                                                                               
   serviceName="kafka";                                                                  
                                                                               
};                                                                                       
                                                                               
Client {                                                                                 
                                                                               
   com.sun.security.auth.module.Krb5LoginModule required                                 
                                                                               
   useTicketCache=true                                                                   
                                                                               
   renewTicket=true                                                                      
                                                                               
   serviceName="zookeeper";                                                              
                                                                               
};                                                                                       
                                                                               
                                    


Regards,
Andrey                                                                                   
                                                 
              



Mime
View raw message