lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From abdul aleem <janaabdulal...@yahoo.com>
Subject Re: Using Lucene to search log files
Date Mon, 11 Dec 2006 15:03:09 GMT
Many thanks to All,

well kind of puzzled because ours is a fast moving log
down to Milliseconds :( as we deal with forex on a
financial system.

Im sure there will be workarounds, actually most of
the time it is enough to search within 2 log files of
1-5MB size, coz we are more intersted in seconds and
minutes than hour as we need to get the logs between
minutes

I had regular expressions in Mind before thinking of
Lucene, but i still feel Lucene should be appropriate
to deal with this, 

I will search the archives for respective issues, Also
have Lucene in Action book i guess it should help


Many thanks again,
Abdul


--- mark harwood <markharw00d@yahoo.co.uk> wrote:

> >>Extend QueryParser to sort this out.
> 
> The latest version in SVN has changed the default
> QueryParser behaviour to use RangeFilters instead of
> RangeQuerys
> 
> 
> 
> ----- Original Message ----
> From: Mike Streeton
> <mike.streeton@ardentiasearch.com>
> To: java-user@lucene.apache.org
> Sent: Monday, 11 December, 2006 1:35:47 PM
> Subject: RE: Using Lucene to search log files
> 
> I would use a RangeFilter instead of using the
> default Boolean query as
> this will always break at some point with Too many
> Boolean clauses.
> Extend QueryParser to sort this out.  As far as
> extracting information
> from log files I would look at creating yourself a
> LogAnalyzer that can
> interpret the contents and indexing it
> appropriately.
> 
> Hope this helps
> 
> Mike
> 
> www.ardentiasearch.com the home of NetSearch
> -----Original Message-----
> From: abdul aleem [mailto:janaabdulaleem@yahoo.com] 
> Sent: 11 December 2006 12:04
> To: java-user@lucene.apache.org
> Subject: Using Lucene to search log files
> 
> Hi All,
> 
> Im a Lucene newbie,
> 
> 
> Requirement : 
> ==============
> a) Build a log viewer tool, search log files for
> keywords and time stamp
> 
> b)  files in production approx 200 logs per day and
> each log file may range from 1MB - 5MB
> 
> Lucene 
> ========
> We wanted to utilize Lucene's search capabilities
> especially search all 200 log files content quickly
> 
> a) Search criteria:
>     i) Timestamp search: Fetch contents between any
> two timestamps 
> 
>    ii) Fetch log file contents for specified keyword
> 
> 
> 
> Query 
> ========
>     a ) Would greatly appreciate if some suggestions
> 
>         whether Lucene will be appropriate tool for
> the requirement ??
> 
> 
>     b) I have tried to use SpanQuery however
> struggling to fetch entire conents e.g. (between two
> timestamps) 
> 
>     c) I had also looked at
> LargeScaleDateRangeProcessing in the wiki, is that a
> right approach for the requirement
> 
> 
> 
>   Any help / suggestion would be greatly
> appreciated,
> 
> 
>   Many thanks in advance,
>   Abdul    
> 
> 
> 
>  
>
________________________________________________________________________
> ____________
> Do you Yahoo!?
> Everyone is raving about the all-new Yahoo! Mail
> beta.
> http://new.mail.yahoo.com
> 
>
---------------------------------------------------------------------
> To unsubscribe, e-mail:
> java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail:
> java-user-help@lucene.apache.org
> 
> 
> 
> 
>
---------------------------------------------------------------------
> To unsubscribe, e-mail:
> java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail:
> java-user-help@lucene.apache.org
> 
> 
> 
> 
> 
> 
> 		
>
___________________________________________________________
> 
> Inbox full of spam? Get leading spam protection and
> 1GB storage with All New Yahoo! Mail.
> http://uk.docs.yahoo.com/nowyoucan.html
> 
>
---------------------------------------------------------------------
> To unsubscribe, e-mail:
> java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail:
> java-user-help@lucene.apache.org
> 
> 



 
____________________________________________________________________________________
Yahoo! Music Unlimited
Access over 1 million songs.
http://music.yahoo.com/unlimited

---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org


Mime
View raw message