lucene-solr-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Antonio Lobato <>
Subject Interesting stuff; Solr as a syslog store.
Date Sat, 13 Feb 2010 02:02:28 GMT
Hey everyone, I don't actually have a question, but I just thought I'd 
share something really cool that I did with Solr for our company.

We run a good amount of servers, well into the several hundreds, and 
naturally we need a way to centralize all of the system logs.  For a 
while we used a commercial solution to centralize and search our logs, 
but they wanted to charge us tens of thousands of dollars for just one 
gigabyte/day more of indexed data.  So I said forget it, I'll write my 
own solution!

We already use Solr for some of our other backend searching systems, so 
I came up with an idea to index all of our logs to Solr.  I wrote a 
daemon in perl that listens on the syslog port, and pointed every single 
system's syslog to forward to this single server.  From there, this 
daemon will write to a Solr indexing server after parsing them into 
fields, such as date/time, host, program, pid, text, etc.  I then wrote 
a cool javascript/ajax web front end for Solr searching, and bam.  Real 
time searching of all of our syslogs from a web interface, for no cost!

Just thought this would be a neat story to share with you all.  I've 
really grown to love Solr, it's something else!


View raw message