lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From abdul aleem <janaabdulal...@yahoo.com>
Subject Re: Using Lucene to search log files
Date Mon, 11 Dec 2006 15:05:35 GMT
Aplogies, forget to mention there are great people
around in this group, they are of great help as well
:):)


--- abdul aleem <janaabdulaleem@yahoo.com> wrote:

> Many thanks to All,
> 
> well kind of puzzled because ours is a fast moving
> log
> down to Milliseconds :( as we deal with forex on a
> financial system.
> 
> Im sure there will be workarounds, actually most of
> the time it is enough to search within 2 log files
> of
> 1-5MB size, coz we are more intersted in seconds and
> minutes than hour as we need to get the logs between
> minutes
> 
> I had regular expressions in Mind before thinking of
> Lucene, but i still feel Lucene should be
> appropriate
> to deal with this, 
> 
> I will search the archives for respective issues,
> Also
> have Lucene in Action book i guess it should help
> 
> 
> Many thanks again,
> Abdul
> 
> 
> --- mark harwood <markharw00d@yahoo.co.uk> wrote:
> 
> > >>Extend QueryParser to sort this out.
> > 
> > The latest version in SVN has changed the default
> > QueryParser behaviour to use RangeFilters instead
> of
> > RangeQuerys
> > 
> > 
> > 
> > ----- Original Message ----
> > From: Mike Streeton
> > <mike.streeton@ardentiasearch.com>
> > To: java-user@lucene.apache.org
> > Sent: Monday, 11 December, 2006 1:35:47 PM
> > Subject: RE: Using Lucene to search log files
> > 
> > I would use a RangeFilter instead of using the
> > default Boolean query as
> > this will always break at some point with Too many
> > Boolean clauses.
> > Extend QueryParser to sort this out.  As far as
> > extracting information
> > from log files I would look at creating yourself a
> > LogAnalyzer that can
> > interpret the contents and indexing it
> > appropriately.
> > 
> > Hope this helps
> > 
> > Mike
> > 
> > www.ardentiasearch.com the home of NetSearch
> > -----Original Message-----
> > From: abdul aleem
> [mailto:janaabdulaleem@yahoo.com] 
> > Sent: 11 December 2006 12:04
> > To: java-user@lucene.apache.org
> > Subject: Using Lucene to search log files
> > 
> > Hi All,
> > 
> > Im a Lucene newbie,
> > 
> > 
> > Requirement : 
> > ==============
> > a) Build a log viewer tool, search log files for
> > keywords and time stamp
> > 
> > b)  files in production approx 200 logs per day
> and
> > each log file may range from 1MB - 5MB
> > 
> > Lucene 
> > ========
> > We wanted to utilize Lucene's search capabilities
> > especially search all 200 log files content
> quickly
> > 
> > a) Search criteria:
> >     i) Timestamp search: Fetch contents between
> any
> > two timestamps 
> > 
> >    ii) Fetch log file contents for specified
> keyword
> > 
> > 
> > 
> > Query 
> > ========
> >     a ) Would greatly appreciate if some
> suggestions
> > 
> >         whether Lucene will be appropriate tool
> for
> > the requirement ??
> > 
> > 
> >     b) I have tried to use SpanQuery however
> > struggling to fetch entire conents e.g. (between
> two
> > timestamps) 
> > 
> >     c) I had also looked at
> > LargeScaleDateRangeProcessing in the wiki, is that
> a
> > right approach for the requirement
> > 
> > 
> > 
> >   Any help / suggestion would be greatly
> > appreciated,
> > 
> > 
> >   Many thanks in advance,
> >   Abdul    
> > 
> > 
> > 
> >  
> >
>
________________________________________________________________________
> > ____________
> > Do you Yahoo!?
> > Everyone is raving about the all-new Yahoo! Mail
> > beta.
> > http://new.mail.yahoo.com
> > 
> >
>
---------------------------------------------------------------------
> > To unsubscribe, e-mail:
> > java-user-unsubscribe@lucene.apache.org
> > For additional commands, e-mail:
> > java-user-help@lucene.apache.org
> > 
> > 
> > 
> > 
> >
>
---------------------------------------------------------------------
> > To unsubscribe, e-mail:
> > java-user-unsubscribe@lucene.apache.org
> > For additional commands, e-mail:
> > java-user-help@lucene.apache.org
> > 
> > 
> > 
> > 
> > 
> > 
> > 		
> >
>
___________________________________________________________
> > 
> > Inbox full of spam? Get leading spam protection
> and
> > 1GB storage with All New Yahoo! Mail.
> > http://uk.docs.yahoo.com/nowyoucan.html
> > 
> >
>
---------------------------------------------------------------------
> > To unsubscribe, e-mail:
> > java-user-unsubscribe@lucene.apache.org
> > For additional commands, e-mail:
> > java-user-help@lucene.apache.org
> > 
> > 
> 
> 
> 
>  
>
____________________________________________________________________________________
> Yahoo! Music Unlimited
> Access over 1 million songs.
> http://music.yahoo.com/unlimited
> 
>
---------------------------------------------------------------------
> To unsubscribe, e-mail:
> java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail:
> java-user-help@lucene.apache.org
> 
> 



 
____________________________________________________________________________________
Do you Yahoo!?
Everyone is raving about the all-new Yahoo! Mail beta.
http://new.mail.yahoo.com

---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org


Mime
View raw message