cloudstack-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Santhosh Edukulla <santhosh.eduku...@gmail.com>
Subject Re: CloudStack + ELK with log4j-jsonevent-layout
Date Sun, 02 Aug 2015 23:46:24 GMT
Thomas,

We had a business requirement of correlating logs (sort of SIEM) from
endpoints, edge devices for threat identification. Few of the applications
were in Java with log4j log configuration, few in other streams. I have
pasted the design we followed to configure and forward the logs to elastic
for indexing and analyzing thereafter, look at it. So, we forwarded it to
logstash server, which is configured with log4j encoder, i believe thats
what you want as well, rather than you trying to explicitly converting logs
to json.

http://www.tiikoni.com/tis/view/?id=7a93bfc

Regards,
Santhosh

On Fri, Jul 31, 2015 at 11:03 PM, Thomas Schneider <
thomas.schneider@euskill.com> wrote:

> I downloaded the jar from
>
> http://central.maven.org/maven2/net/logstash/log4j/jsonevent-layout/1.7/jsonevent-layout-1.7.jar
>
> Put it in /usr/share/cloudstack-management/webapps/client/WEB-INF/lib/
>
> then edited /etc/cloudstack/management/log4j-cloud.xml
>
> From:
>
>    <appender name="FILE"
> class="org.apache.log4j.rolling.RollingFileAppender">
>       <param name="Append" value="true"/>
>       <param name="Threshold" value="TRACE"/>
>       <rollingPolicy
> class="org.apache.log4j.rolling.TimeBasedRollingPolicy">
>         <param name="FileNamePattern"
>
> value="/var/log/cloudstack/management/management-server.log.%d{yyyy-MM-dd}.gz"/>
>         <param name="ActiveFileName"
> value="/var/log/cloudstack/management/management-server.log"/>
>       </rollingPolicy>
>       <layout class="org.apache.log4j.EnhancedPatternLayout">
>          <param name="ConversionPattern" value="%d{ISO8601} %-5p
> [%c{1.}] (%t:%x) %m%n"/>
>    </appender>
>
> To:
>    <appender name="FILE"
> class="org.apache.log4j.rolling.RollingFileAppender">
>       <param name="Append" value="true"/>
>       <param name="Threshold" value="TRACE"/>
>       <rollingPolicy
> class="org.apache.log4j.rolling.TimeBasedRollingPolicy">
>         <param name="FileNamePattern"
>
> value="/var/log/cloudstack/management/management-server.log.%d{yyyy-MM-dd}.gz"/>
>         <param name="ActiveFileName"
> value="/var/log/cloudstack/management/management-server.log"/>
>       </rollingPolicy>
>       <layout class="net.logstash.log4j.JSONEventLayoutV1" />
>    </appender>
>
> but after that I don't have log anymore.
>
> I also tryed to:
> mkdir -p /root/classpath/
> cp jsonevent-layout-1.7.jar /root/classpath
> vi /etc/environement
> add: CLASSPATH="/root/classpath"
> source /etc/environement
>
> but i have the same result.
>
> If someone can advice me ?
>
>
> Le 30/07/2015 21:36, Thomas Schneider a écrit :
> > Hello,
> >
> > I would like to setup ELK stack to monitor CloudStack Log.
> > I have already setup a central Elastisearch + Logstach + Kibana server
> > who receive logs from all my cloudstack management server via
> > logstach-forwader and it work pretty well with the standart system's log
> > file like /var/log/syslog etc... because they can be easyly parsed by
> > logstach's grok filter.
> >
> > But the main problem I have, is I didn't find a good technique to parse
> > cloudstack log file.
> >
> > However i founded a plugin for log4j who's called log4j-jsonevent-layout
> > that can output the cloudstack log in json and the json log are easy to
> > parse for logstache, but a dindn't found how to setup this plugin.
> >
> > So howto setup log4j-jsonevent-layout witch cloudstack ?
> > If someone can advise me on this issue.
> >
> > Regards,
>
> --
> *Thomas Schneider*
> Directeur des Opérations
> Euskill SARL
> Web: www.euskill.com
> Mobile: +33 (0)6 19 26 47 76
> Mail: thomas.schneider@euskill.com
> 5 rue de Phalsbourg
> F-67000 Strasbourg
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message