lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Erick Erickson" <erickerick...@gmail.com>
Subject Re: Using Lucene to search log files
Date Mon, 11 Dec 2006 15:23:59 GMT
Then you really want to look at the classes that do the work with filters if
you require milliseconds. You should be just fine....

On 12/11/06, abdul aleem <janaabdulaleem@yahoo.com> wrote:
>
> Many thanks to All,
>
> well kind of puzzled because ours is a fast moving log
> down to Milliseconds :( as we deal with forex on a
> financial system.
>
> Im sure there will be workarounds, actually most of
> the time it is enough to search within 2 log files of
> 1-5MB size, coz we are more intersted in seconds and
> minutes than hour as we need to get the logs between
> minutes
>
> I had regular expressions in Mind before thinking of
> Lucene, but i still feel Lucene should be appropriate
> to deal with this,
>
> I will search the archives for respective issues, Also
> have Lucene in Action book i guess it should help
>
>
> Many thanks again,
> Abdul
>
>
> --- mark harwood <markharw00d@yahoo.co.uk> wrote:
>
> > >>Extend QueryParser to sort this out.
> >
> > The latest version in SVN has changed the default
> > QueryParser behaviour to use RangeFilters instead of
> > RangeQuerys
> >
> >
> >
> > ----- Original Message ----
> > From: Mike Streeton
> > <mike.streeton@ardentiasearch.com>
> > To: java-user@lucene.apache.org
> > Sent: Monday, 11 December, 2006 1:35:47 PM
> > Subject: RE: Using Lucene to search log files
> >
> > I would use a RangeFilter instead of using the
> > default Boolean query as
> > this will always break at some point with Too many
> > Boolean clauses.
> > Extend QueryParser to sort this out.  As far as
> > extracting information
> > from log files I would look at creating yourself a
> > LogAnalyzer that can
> > interpret the contents and indexing it
> > appropriately.
> >
> > Hope this helps
> >
> > Mike
> >
> > www.ardentiasearch.com the home of NetSearch
> > -----Original Message-----
> > From: abdul aleem [mailto:janaabdulaleem@yahoo.com]
> > Sent: 11 December 2006 12:04
> > To: java-user@lucene.apache.org
> > Subject: Using Lucene to search log files
> >
> > Hi All,
> >
> > Im a Lucene newbie,
> >
> >
> > Requirement :
> > ==============
> > a) Build a log viewer tool, search log files for
> > keywords and time stamp
> >
> > b)  files in production approx 200 logs per day and
> > each log file may range from 1MB - 5MB
> >
> > Lucene
> > ========
> > We wanted to utilize Lucene's search capabilities
> > especially search all 200 log files content quickly
> >
> > a) Search criteria:
> >     i) Timestamp search: Fetch contents between any
> > two timestamps
> >
> >    ii) Fetch log file contents for specified keyword
> >
> >
> >
> > Query
> > ========
> >     a ) Would greatly appreciate if some suggestions
> >
> >         whether Lucene will be appropriate tool for
> > the requirement ??
> >
> >
> >     b) I have tried to use SpanQuery however
> > struggling to fetch entire conents e.g. (between two
> > timestamps)
> >
> >     c) I had also looked at
> > LargeScaleDateRangeProcessing in the wiki, is that a
> > right approach for the requirement
> >
> >
> >
> >   Any help / suggestion would be greatly
> > appreciated,
> >
> >
> >   Many thanks in advance,
> >   Abdul
> >
> >
> >
> >
> >
> ________________________________________________________________________
> > ____________
> > Do you Yahoo!?
> > Everyone is raving about the all-new Yahoo! Mail
> > beta.
> > http://new.mail.yahoo.com
> >
> >
> ---------------------------------------------------------------------
> > To unsubscribe, e-mail:
> > java-user-unsubscribe@lucene.apache.org
> > For additional commands, e-mail:
> > java-user-help@lucene.apache.org
> >
> >
> >
> >
> >
> ---------------------------------------------------------------------
> > To unsubscribe, e-mail:
> > java-user-unsubscribe@lucene.apache.org
> > For additional commands, e-mail:
> > java-user-help@lucene.apache.org
> >
> >
> >
> >
> >
> >
> >
> >
> ___________________________________________________________
> >
> > Inbox full of spam? Get leading spam protection and
> > 1GB storage with All New Yahoo! Mail.
> > http://uk.docs.yahoo.com/nowyoucan.html
> >
> >
> ---------------------------------------------------------------------
> > To unsubscribe, e-mail:
> > java-user-unsubscribe@lucene.apache.org
> > For additional commands, e-mail:
> > java-user-help@lucene.apache.org
> >
> >
>
>
>
>
>
> ____________________________________________________________________________________
> Yahoo! Music Unlimited
> Access over 1 million songs.
> http://music.yahoo.com/unlimited
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message