karaf-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Achim Nierbeck <bcanh...@googlemail.com>
Subject Re: Switch karaf decanter to EventAdmin
Date Tue, 17 Mar 2015 21:28:58 GMT
2015-03-17 22:03 GMT+01:00 Christian Schneider <chris@die-schneider.net>:

> On 17.03.2015 21:16, Achim Nierbeck wrote:
>> Hi,
>> I did take a look at the event admin solution.
>> One question up front. Why do we need the logging collector with the event
>> admin solution? If eventadmin is present all log entries should/could be
>> posted directly via the eventadmin.
> I like the idea. How would that work? With a special log4j appender or
> does pax-logging already support this?
> Btw. one reason for the log collector is to translate from the
> PaxLoggingEvent to a simple Map<String, Object>. Another thing to not is
> that the MDC values are copied
> into the key "MDC" and contain a sub map. Does it already work like this?
> Basically what I thought was to have the convention thar an EventAdmin
> Event would only contain properties of simple types or of Map<String,
> Object> where the value is again a simple type. For the most part this is
> how EventAdmin events should be done according to the spec.

As long as it works right now, good enough.
Afaik Pax Logging does either already log out via Events, or it just needs
to be added to the configuration file.

>> Regarding Topics and Indices, I don't think its a good idea to create
>> multiple indices, but the topic can be used differently.
>> Instead of sending karaf_events
> I agree to not have separate indices. If elastic search can not correlate
> between indices it is better to have them in one index.
> Some people might still want it for some reason but I think the major case
> is to put all into the same index. So I think we can delay providing
> configs for several indexes until there is a first good use case.

Indices are usually time-based not content based (like the event topics)
So it just makes sense to maybe have either an index for each hour (if the
data is to much already)
Or an aggregated Index for a week.

>> client.prepareIndex(indexName, "karaf_event").setSource(jsonObject
>> .toString()).execute().actionGet();
>> the topic could be used instead
>> client.prepareIndex(indexName, event.getTopic()).setSource(jsonObject
>> .toString()).execute().actionGet();
>> That should be more like what you where expecting to achieve with the
>> index.
> So you would use the Topic as the type. One thing to consider here is that
> I currently translate the logger name into a topic name. This has the
> advantage that it is quite easy in EventAdmin to filter for a logger name
> including all sub names.  If you create the type from this that would make
> lots of different types. I am not sure what would happen in Elastic Search
> with that.

In the standard ELK scenario for analysing log data, you have different
logfiles that produces those entries. Therefore those logfiles usually have
their own type. Like for example Access and Syslog.

> As far as I understood a type means that each type has different
> properties. So maybe we can derive the type from a different field. For
> example we could have the type as a separate field that the collectors set.
> So for example for JMX each bean could be a different type as they all have
> different properties.
Yes that could be the next enhancement for the JMX Collector.

> For logs I am not sure. Most log entries will be the same but as MDC comes
> into play users can build their own kind of types. Not sure how to handle
> that. Currently I simply put all into the same index in the end so the MDC
> values are all there but there is not futher structure. Maybe this is
> completely ok for how elastic search works.

yes, afaik this is sufficient already.
If you only use the JMX Collector you'll see that EL is already capable to
map different mappings to that type already.

regards, Achim

> Thanks for the feedback Achim.
> Christian
> --
> Christian Schneider
> http://www.liquid-reality.de
> Open Source Architect
> http://www.talend.com


Apache Member
Apache Karaf <http://karaf.apache.org/> Committer & PMC
OPS4J Pax Web <http://wiki.ops4j.org/display/paxweb/Pax+Web/> Committer &
Project Lead
blog <http://notizblog.nierbeck.de/>
Co-Author of Apache Karaf Cookbook <http://bit.ly/1ps9rkS>

Software Architect / Project Manager / Scrum Master

  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message