cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hiller, Dean" <Dean.Hil...@nrel.gov>
Subject Re: logging servers? any interesting in one for cassandra?
Date Tue, 06 Nov 2012 15:19:27 GMT
Sure, in our playing around, we have an awesome log back configuration for
development time only that shows warning, severe in red in eclipse and
let's you click on every single log taking you right to the code that
logged it…(thought you might enjoy it)...

https://github.com/deanhiller/playorm/blob/master/input/javasrc/logback.xml


The java appender is here(called CassandraAppender)
https://github.com/deanhiller/playorm/tree/master/input/javasrc/com/alvazan
/play/logging


The AsyncAppender there is different then log backs in that it allows
bursting but once reaches the limit, it essentially becomes synchronous
again which allows us to not drop logs like log backs and allow for bursts
of performance

The CircularBufferAppender is an inmemory buffer that flushes all logs X
level and above to child appender when a warning or severe happens where X
is configurable.  

We have only tested out the CassandraAppender at this point.  Right now
you have to call CassandraAppender.setFactory to set the
NoSqlEntityManager factory to set it.  It creates a LogEvent rows as well
as an index on the session and partitions by the first two characters of
the web session id so there is an index per partition.  This allows us to
the look at a single web session of a user.  The only thing I don't like
is we have to do a read when updating the index to be able to delete old
values in the index(ick), but I couldn't figure any other way around that.

Also, if you have high event rates, there is a MDCLevelFilter so you can
tag the MDC with something like user=__program__ and ignore all logs for
him unless they are warning logs which we use to limit the logs from just
being huge.

Later,
Dean


On 11/6/12 6:32 AM, "Brian O'Neill" <bone@alumni.brown.edu> wrote:

>Nice DeanŠ
>
>I'm not so sure we would run the server, but we'd definitely be interested
>in the logback adaptor.
>(We would then just access the data via Virgil (over REST), with a thin
>javascript UI)
>
>Let me/us know if you end up putting it out there.  We intend centralize
>logging sometime over the next few months.
>
>-brian
>
>---
>Brian O'Neill
>Lead Architect, Software Development
>Health Market Science
>The Science of Better Results
>2700 Horizon Drive € King of Prussia, PA € 19406
>M: 215.588.6024 € @boneill42 <http://www.twitter.com/boneill42>  €
>healthmarketscience.com
>
>This information transmitted in this email message is for the intended
>recipient only and may contain confidential and/or privileged material. If
>you received this email in error and are not the intended recipient, or
>the person responsible to deliver it to the intended recipient, please
>contact the sender at the email above and delete this email and any
>attachments and destroy any copies thereof. Any review, retransmission,
>dissemination, copying or other use of, or taking any action in reliance
>upon, this information by persons or entities other than the intended
>recipient is strictly prohibited.
> 
>
>
>
>
>
>
>On 11/1/12 10:33 AM, "Hiller, Dean" <Dean.Hiller@nrel.gov> wrote:
>
>>2 questions
>>
>> 1.  What are people using for logging servers for their web tier
>>logging?
>> 2.  Would anyone be interested in a new logging server(any programming
>>language) for web tier to log to your existing cassandra(it uses up disk
>>space in proportion to number of web servers and just has a rolling
>>window of logs along with a window of threshold dumps)?
>>
>>Context for second question: I like less systems since it is less
>>maintenance/operations cost and so yesterday I quickly wrote up some log
>>back appenders which support (SLF4J/log4j/jdk/commons libraries) and send
>>the logs from our client tier into cassandra.  It is simply a rolling
>>window of logs so the space used in cassandra is proportional to the
>>amount of web  servers I have(currently, I have 4 web servers).  I am
>>also thinking about adding warning type logging such that on warning, the
>>last N logs info and above are flushed along with the warning so
>>basically two rolling windows.  Then in the GUI, it simply shows the logs
>>and if you click on a session, it switches to a view with all the logs
>>for that session(no matter which server since in our cluster the session
>>switches servers on every request since we are statelessŠ.our session id
>>is in the cookie).
>>
>>Well, let me know if anyone is interested and would actually use such a
>>thing and if so, we might create a server around it.
>>
>>Thanks,
>>Dean
>
>

Mime
View raw message