lucene-solr-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Kay Kay <>
Subject Re: Interesting stuff; Solr as a syslog store.
Date Sat, 13 Feb 2010 11:26:42 GMT
Thanks Antonio for sharing this.

I believe this could be one of the interesting case studies for Solr In 
Action, if you are interested in sharing a bit more - I am sure the 
authors would be more interested for upcoming revisions.

  K K.

On 02/12/2010 06:02 PM, Antonio Lobato wrote:
> Hey everyone, I don't actually have a question, but I just thought I'd 
> share something really cool that I did with Solr for our company.
> We run a good amount of servers, well into the several hundreds, and 
> naturally we need a way to centralize all of the system logs.  For a 
> while we used a commercial solution to centralize and search our logs, 
> but they wanted to charge us tens of thousands of dollars for just one 
> gigabyte/day more of indexed data.  So I said forget it, I'll write my 
> own solution!
> We already use Solr for some of our other backend searching systems, 
> so I came up with an idea to index all of our logs to Solr.  I wrote a 
> daemon in perl that listens on the syslog port, and pointed every 
> single system's syslog to forward to this single server.  From there, 
> this daemon will write to a Solr indexing server after parsing them 
> into fields, such as date/time, host, program, pid, text, etc.  I then 
> wrote a cool javascript/ajax web front end for Solr searching, and 
> bam.  Real time searching of all of our syslogs from a web interface, 
> for no cost!
> Just thought this would be a neat story to share with you all.  I've 
> really grown to love Solr, it's something else!
> Thanks,
> -Antonio

View raw message