eagle-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (EAGLE-888) Application submitted to Storm is always shown as “HBaseAuditLogApp”
Date Fri, 10 Feb 2017 19:44:41 GMT

    [ https://issues.apache.org/jira/browse/EAGLE-888?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15861748#comment-15861748
] 

ASF GitHub Bot commented on EAGLE-888:
--------------------------------------

Github user DadanielZ commented on the issue:

    https://github.com/apache/eagle/pull/796
  
    @wujinhu thanks 👍 , this issue is solved.
    
    But I come up with another one when tring to view the alerts from UI:
    
    After configuring the "application-debug.conf" properly with *"jdbc"* as the storage type,
I started eagle from Intellij successfully and started alert-engine and hdfsAuditLogApp smoothly,
then I created a hdfs policy.
    
    I saw alerts are generated and saved in my mysql database. 
    
    But when I tried to view the alerts from UI, ERROR occurs:
    ~~~
    ERROR [2017-02-10 01:19:40,433] org.apache.eagle.service.generic.GenericEntityServiceResource:
unknown storage type: jdbc, support: [hbase]
    ! org.apache.eagle.storage.exception.IllegalDataStorageTypeException: unknown storage
type: jdbc, support: [hbase]
    ~~~
    
    During debugging I see `storageServiceProviders` only contains `hBaseStorage`, but no
`jdbcStorage`. Could you tell me the reason or give me a hint on which module to look into?

    
    Thanks and regards,
    Da
    
     Here is the configuration file I used:
    
    ~~~
    # Licensed to the Apache Software Foundation (ASF) under one or more
    # contributor license agreements.  See the NOTICE file distributed with
    # this work for additional information regarding copyright ownership.
    # The ASF licenses this file to You under the Apache License, Version 2.0
    # (the "License"); you may not use this file except in compliance with
    # the License.  You may obtain a copy of the License at
    #
    #    http://www.apache.org/licenses/LICENSE-2.0
    #
    # Unless required by applicable law or agreed to in writing, software
    # distributed under the License is distributed on an "AS IS" BASIS,
    # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
    # See the License for the specific language governing permissions and
    # limitations under the License.
    
    # ---------------------------------------------
    # Eagle REST Web Service Configuration
    # ---------------------------------------------
    service {
      env = "testing"
      host = "localhost"
      port = 9090
      username = "admin"
      password = "secret"
      readTimeOutSeconds = 60
      context = "/rest"
      timezone = "UTC"
    }
    
    zookeeper {
      zkQuorum = "localhost:2181"
      zkSessionTimeoutMs : 15000
      zkRetryTimes : 3
      zkRetryInterval : 20000
    }
    
    # ---------------------------------------------
    # Eagle Deep Storage Configuration
    # ---------------------------------------------
    
    storage {
      type = "jdbc"
      jdbc {
        adapter = "mysql"
        username = "root"
        password = root
        database = eagle
        connectionUrl = "jdbc:mysql://localhost:3306/eagle"
        connectionProps = "encoding=UTF-8"
        driverClass = "com.mysql.jdbc.Driver"
        connectionMax = 8
      }
    }
    
    # ---------------------------------------------
    # Eagle Metadata Store Configuration
    # ---------------------------------------------
    
    
    metadata {
      store = "org.apache.eagle.metadata.store.jdbc.JDBCMetadataStore"
      jdbc {
        username = root
        password = root
        driverClassName = com.mysql.jdbc.Driver
        database = "eagle"
        connection = "jdbc:mysql://localhost:3306/eagle?user=root"
      }
    }
    # ---------------------------------------------
    # Eagle Application Configuration
    # ---------------------------------------------
    application {
      stream {
        provider = org.apache.eagle.app.messaging.KafkaStreamProvider
      }
      storm {
        nimbusHost = "localhost"
        nimbusThriftPort = 6627
      }
      updateStatus: {
        initialDelay: 10
        period: 10
      }
      healthCheck {
        initialDelay = 30
        period = 60
        publisher {
          publisherImpl = org.apache.eagle.app.service.impl.ApplicationHealthCheckEmailPublisher
          dailySendHour = 11
          mail.smtp.host = "mail.host.com"
          mail.smtp.port = 25
          mail.smtp.recipients = "someone@email.com"
          mail.smtp.subject = "Eagle Application Health Check"
          mail.smtp.template = "HealthCheckTemplate.vm"
        }
      }
      mailService {
        mailSmtpServer = "",
        mailSmtpPort = 25,
        mailSmtpAuth = "false"
        //mailSmtpConn = "plaintext",
        //mailSmtpUsername = ""
        //mailSmtpPassword = ""
        //mailSmtpDebug = false
      }
      dailyJobReport {
        reportHourTime: 1
        reportPeriodInHour: 12
        numTopUsers : 10
        jobOvertimeLimitInHour: 6
        subject: "Job Report For 12 hours"
        recipients: "nobody@abc.com"
        template: "JobReportTemplate.vm"
      }
    }
    
    # ---------------------------------------------
    # Eagle Alert Engine Configuration
    # ---------------------------------------------
    
    # Coordinator Configuration
    coordinator {
      policiesPerBolt = 5
      boltParallelism = 5
      policyDefaultParallelism = 5
      boltLoadUpbound = 0.8
      topologyLoadUpbound = 0.8
      numOfAlertBoltsPerTopology = 5
      zkConfig {
        zkQuorum = "localhost:2181"
        zkRoot = "/alert"
        zkSessionTimeoutMs = 10000
        connectionTimeoutMs = 10000
        zkRetryTimes = 3
        zkRetryInterval = 3000
      }
      metadataService {
        host = "localhost",
        port = 9090,
        context = "/rest"
      }
      metadataDynamicCheck {
        initDelayMillis = 1000
        delayMillis = 30000
        stateClearPeriodMin = 1440
        stateReservedCapacity = 100
      }
    }
    
    
    ~~~
    
    
     


> Application submitted to  Storm is always shown as “HBaseAuditLogApp”
> ---------------------------------------------------------------------
>
>                 Key: EAGLE-888
>                 URL: https://issues.apache.org/jira/browse/EAGLE-888
>             Project: Eagle
>          Issue Type: Bug
>          Components: Core::App Engine
>    Affects Versions: v0.5.0
>            Reporter: DanielZhou
>            Assignee: DanielZhou
>
> *Issue*:
> Steps to reproduce:
> - Started application from Eagle UI (eg: alert engine)
> - Go to Storm UI, topology name is shown as *"HBaseAuditLogApp"* 
> *Reason*:
> In the constructor function of class *"ApplicationAction"*:
> {quote}
> this.effectiveConfig = ConfigFactory.parseMap(executionConfig)
>                     .withFallback(serverConfig)
>                     .withFallback(ConfigFactory.parseMap(metadata.getContext()))
> {quote}
> According to the java doc of [withFallBack(theOther)|http://typesafehub.github.io/config/latest/api/com/typesafe/config/Config.html#withFallback-com.typesafe.config.ConfigMergeable-]
:
> {quote}
> Returns a new value computed by merging this value with another, with keys in this value
"winning" over the other one.
> {quote}
> As a result, "serverConfig" will win over "ConfigFactory.parseMap(metadata.getContext())"
which means the default "ConfigString(appId="HBaseAuditApp")" and "ConfigString(siteId="testSite")"
will win over the meta data of the user's topology.
> *Fix*:
> Change the order of "withFallBack" to:
> {quote}
> this.effectiveConfig = ConfigFactory.parseMap(executionConfig)
>                     .withFallback(ConfigFactory.parseMap(metadata.getContext()))
>                     .withFallback(serverConfig)
> {quote}



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

Mime
View raw message