metron-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ryan Merriman <merrim...@gmail.com>
Subject Re: Unable to see alerts in metron alert UI with solr
Date Thu, 23 Aug 2018 13:42:18 GMT
You can pass credentials in curl with the -u flag:  curl -u user:password
-X GET --header 'Accept: application/json' 'http://{rest
host}:8082/api/v1/sensor/indexing/config/list/indices/solr'.  I prefer to
use Swagger.

We need to dig into the 500 response.  An error message should be included
in the response or you can try running it separately in Swagger.  I would
also check the log at /var/log/metron/metron-rest.log.



On Thu, Aug 23, 2018 at 6:35 AM Anil Donthireddy <Anil.Donthireddy@sstech.us>
wrote:

> @Ryan: I tried the api call from swagger UI. I can see the list of indices
> is being returned fine as present in Solr.
>
>
>
> *From:* Anil Donthireddy [mailto:Anil.Donthireddy@sstech.us]
> *Sent:* Thursday, August 23, 2018 5:03 PM
> *To:* user@metron.apache.org
> *Subject:* RE: Unable to see alerts in metron alert UI with solr
>
>
>
> Hi,
>
>
>
> Thanks a lot for the inputs.
>
>
>
> @Stefan: We don’t have ranger setup for access permissions.
>
>
>
> @Ryan: Below are the issues I observed from your suggestion
>
>                 For the search request in network tab, I see the response
> is “500 Internal Server Error”
>
>                 In the request payload I see all the params are emtpy as
> below.
>
>                                 {indices: [], facetFields: [], query:
> "*", from: 0, size: 25}
>
>        When I try to run the curl command, I am getting unauthorized
> error as below. How can I pass credentials for authentication along with
> the request or how can I authenticate via curl command and store the
> jessionid as cookie and pass through your curl command?
>
>               {"timestamp":"2018-08-23
> 11:13:22","status":401,"error":"Unauthorized","message":"Unauthorized","path":"/api/v1/sensor/indexing/config/list/indices/solr"}
>
>
>
>
>
> Thanks,
>
> Anil.
>
>
>
> *From:* Ryan Merriman [mailto:merrimanr@gmail.com <merrimanr@gmail.com>]
> *Sent:* Thursday, August 23, 2018 3:41 AM
> *To:* user@metron.apache.org
> *Subject:* Re: Unable to see alerts in metron alert UI with solr
>
>
>
> The first thing I would do is figure out which indices are included in the
> Alerts UI.  If you open the network tab in your browser there should be a
> "search" request sent to REST.  There is an "indices" property that will
> have a list of indices include in the search.  If it's empty ("[]") then
> this call should tell you what the default indices are:
>
>
>
> curl -X GET --header 'Accept: application/json' 'http://{rest
> host}:8082/api/v1/sensor/indexing/config/list/indices/solr'
>
>
>
> Verify your collection is in the list and we can go from there.
>
>
>
> On Wed, Aug 22, 2018 at 4:47 PM Stefan Kupstaitis-Dunkler <
> stefan.dun@gmail.com> wrote:
>
> Hey Anil,
>
>
>
> do you use any kind of policies to prevent access by the metron user? Do
> you use Ranger to manage access policies?
>
>
>
> The user running the Metron REST service (normally "metron") needs to have
> access to all collections.
>
>
>
> Check the (Ranger) audit log to see if access to the collections is
> denied/allowed.
>
>
>
> If you use Ranger + Solr with Metron you might run into a current Solr
> plugin bug. I've run into that a few weeks ago and provided a workaround
> here:
>
>
>
>
> https://community.hortonworks.com/articles/203326/searching-in-multiple-collections-in-one-query-wit.html
> <https://linkprotect.cudasvc.com/url?a=https%3a%2f%2fcommunity.hortonworks.com%2farticles%2f203326%2fsearching-in-multiple-collections-in-one-query-wit.html&c=E,1,9dD6y1u3ennoQvuZCZ45l9VNmxiTrI6YNYlhGiC0_1XXX0pMj7qHTj6yBi6drFMrosDMlFfiJthaQqZTUjr6MVXrJwuRPuh_eNO6-s9QM6lnTnhpPaw,&typo=1>
>
>
>
> Best,
>
> Stefan
>
>
>
>
>
> On Wed, Aug 22, 2018, 18:40 Anil Donthireddy <Anil.Donthireddy@sstech.us>
> wrote:
>
> Yes, necessary two fields are set properly as below
>
>
>
> "source.type.field" : "source.type",
>
>   "threat.triage.score.field" : "threat.triage.score"
>
>
>
>
>
> *From:* Anand Subramanian [mailto:asubramanian@hortonworks.com]
> *Sent:* Wednesday, August 22, 2018 10:07 PM
> *To:* user@metron.apache.org
> *Subject:* Re: Unable to see alerts in metron alert UI with solr
>
>
>
> Hey Anil,
>
>
>
> You might also want to check if the “source.type.field” is set to
> “source.type” (and NOT source:type) in global config.
>
>
>
> See:
>
>
> https://github.com/apache/metron/blob/master/metron-interface/metron-alerts/README.md#global-configuration-properties
>
>
>
> HTH,
>
> Anand
>
>
>
> *From: *Anil Donthireddy <Anil.Donthireddy@sstech.us>
> *Reply-To: *"user@metron.apache.org" <user@metron.apache.org>
> *Date: *Wednesday, August 22, 2018 at 10:04 PM
> *To: *"user@metron.apache.org" <user@metron.apache.org>
> *Subject: *RE: Unable to see alerts in metron alert UI with solr
>
>
>
> Hi Stefan,
>
>
>
> I can see the events being written to solr collection. There are no issues
> upto storing data to Solr. There are events which have is_alert=true. But
> in the metron alerts UI I dint see any events.
>
>
>
> Thanks,
>
> Anil.
>
>
>
> *From:* Stefan Kupstaitis-Dunkler [mailto:stefan.dun@gmail.com
> <stefan.dun@gmail.com>]
> *Sent:* Wednesday, August 22, 2018 9:52 PM
> *To:* user@metron.apache.org
> *Subject:* Re: Unable to see alerts in metron alert UI with solr
>
>
>
> Hi Anil,
>
>
>
> The alerts UI just queries your Solr collections and displays them
>
>
>
> Things you could do:
>
> ?         You could check the collection the event is supposed to be
> directly
>
> ?         If it’s not there you could check the Solr “error” collection.
>
> ?         Use the Kafka console consumer and
>
> o   check the parser topic (you define the name in the management ui) if
> events are being written to it
>
> o   check the “enrichments” topic if events are being written to it
>
> o   check the “indexing” topic if events are being written to it.
>
>
>
> Hope that helps as a start.
>
>
>
> Best,
>
> Stefan
>
>
>
>
>
>
>
>
>
> *From: *Anil Donthireddy <Anil.Donthireddy@sstech.us>
> *Reply-To: *"user@metron.apache.org" <user@metron.apache.org>
> *Date: *Wednesday, 22. August 2018 at 18:07
> *To: *"user@metron.apache.org" <user@metron.apache.org>
> *Subject: *Unable to see alerts in metron alert UI with solr
>
>
>
> Hi,
>
>
>
> I have created a collection in solr for my source as documented in
> metron-slor readme in git to enable solr and writing data to solr. I see
> the data is being loaded to Solr for my new source and is_alert is set to
> tru. But still I am not seeing any data in metron alerts UI.
>
>
>
> I checked the metron alerts UI logs, but could not find any error logs. I
> am blocked to proceed further to display alerts in the metron alerts ui.
> May I get any suggestions to debug the issue.
>
>
>
> Thanks,
>
> Anil.
>
> --
>
> Stefan Kupstaitis-Dunkler
>
> https://datahovel.com/
> <https://linkprotect.cudasvc.com/url?a=https%3a%2f%2fdatahovel.com%2f&c=E,1,rgB6RnyVWsWRLyz4Pfv-ZJ0kCNc2qsDK9f0qPgKhSLPGI8cUtFb3JzjKtCJ0C5P5533BhkBS_k9oAum3K-N7e8fYNQHryxJrRnCjR5uHcw,,&typo=1>
>
> https://www.meetup.com/Hadoop-User-Group-Vienna/
>
> https://twitter.com/StefanDunkler
>
>

Mime
View raw message