lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Naimdjon Takhirov <tna...@yahoo.com>
Subject Vedr. Re: Design question [too many fields?]
Date Fri, 01 Jul 2005 19:16:40 GMT
Guys, thanks for your inputs.
I think the solution Mark has suggested does solves
the problem in an acceptable way. Its actually gonna
be a little better than the solution the customer is
has right now.
Apart from the availability we have to also check if
there is any price for room units saved in database
since the customer has a flexible pricing system
(discounts in weekends, season prices and so on) so
its very important that the availability/price
information is quite up-to-date.
I implemented it with the filter(Mark: there were
about 4900 room units which I think is OK as far as
memory concerned) and instead of 10 secs searching for
a period of two weeks it takes about 3 seconds now.
Still we have optimization work to do.

Naimdjon

--- Chris Lu <chris.lu@gmail.com> skrev:

> Erik,  Mark and Naimdjon, Sorry I totally
> misunderstood the question,
> of multiple dates for a Document. I came to agree
> with Erik and Mark
> on this problem.
> 
> My head was thinking to find a generic solution to
> Lucene's
> limitation: The TooManyClauses problem when using
> RangeQuery and there
> are more than 1024 values. It should be another
> thread.
> 
> -- 
> Chris Lu
> ---------------------
> Full-Text Search on Any Database
> http://www.dbsight.net
> 
> 
> On 7/1/05, Erik Hatcher <erik@ehatchersolutions.com>
> wrote:
> > 
> > On Jun 30, 2005, at 11:27 PM, Chris Lu wrote:
> > > Mark, your suggestion will incur another trip to
> the database. And
> > > if the search results is large, filtering in DB
> by pk is not really
> > > good.
> > 
> > Chris - I disagree with that last comment.  It can
> be a great
> > solution when the filter is cached.  Certainly
> building a filter for
> > every search would be inefficient, but filters are
> really best when
> > cached.
> > 
> > > Erik, your original "date" field is good when
> there is not many
> > > dates(<1024) in the database. Otherwise, Range
> Query can not handle
> > > it.
> > 
> > Not quite correct... it would not matter how many
> dates were in the
> > index/database, only how many were within the
> range used by
> > RangeQuery.  The original requirement was a years
> worth of days,
> > which at most would be 366 days and I suspect
> someone looking for
> > hotel room availability would be narrowing things
> down to a week or
> > month.
> > 
> > > My suggestion is, use "year" + "month" + "day"
> three fields to
> > > store date. And when searching, for example, any
> date that's
> > > greater than 2005-06-30, you can use this query
> to search: ( year >
> > > 2005 ) or  ( year=2005 and month>=6) or (
> year=2005 and month=6 and
> > > day > 30 ).
> > > It's a combination of BooleanQuery, TermQuery,
> and RangeQuery.
> > 
> > How would you represent multiple dates for a
> Document using that
> > scheme?  Wasn't that one of the original
> requirements?
> > 
> > > This may seem cumbersome, but it can save one
> trip to database, and
> > > circumvent Lucene's limitation.
> > 
> > One trip to the DB *once* with the results cached
> is mighty
> > inexpensive in the grand scheme of things.  Mark's
> point is something
> > I agree with (and wrote about in the custom filter
> example in Lucene
> > in Action) - some information makes good sense to
> stay in a
> > relational database when its too volatile to put
> in a Lucene index.
> > Building a filter to access a DB, with the results
> cached is a good
> > solution to the specified problem, I think. 
> Certainly there are many
> > ways to solve it though.
> > 
> >      Erik
> > 
> > 
> > >
> > > Chris Lu
> > > http://www.dbsight.net
> > >
> > > Erik Hatcher wrote:
> > >
> > >
> > >> I second Mark's suggestion over the alternative
> I posted.  My
> > >> alternative was merely to invert the field
> structure originally
> > >> described, but using a Filter for the volatile
> information is wiser.
> > >>
> > >>     Erik
> > >>
> > >> On Jun 29, 2005, at 9:58 AM, mark harwood
> wrote:
> > >>
> > >>
> > >>> Presumably there is also a free-text element
> to the
> > >>> search or you wouldn't be using Lucene.
> > >>>
> > >>> Multiple fields is not the way to go.
> > >>> A single Lucene field could contain multiple
> terms (
> > >>> the available dates) but I still don't think
> that's
> > >>> the best solution.
> > >>> The availability info is likely to be pretty
> volatile
> > >>> and you always want up-to-date info so I would
> prefer
> > >>> to hit a database for this. If you keep a DB
> primary
> > >>> key to Lucene doc id look-up cached in memory
> you can
> > >>> quickly construct a Lucene filter from the
> database
> > >>> results and therefore only show Lucene results
> for
> > >>> available rooms.
> > >>>
> > >>> Cheers
> > >>> Mark
> > >>>
> > >>>
> > >>>
> > >>>
>
___________________________________________________________
> > >>> How much free photo storage do you get? Store
> your holiday
> > >>> snaps for FREE with Yahoo! Photos
> http://uk.photos.yahoo.com
> > >>>
> > >>>
>
--------------------------------------------------------------------
> > >>> -
> > >>> To unsubscribe, e-mail:
> java-user-unsubscribe@lucene.apache.org
> > >>> For additional commands, e-mail:
> java-user-help@lucene.apache.org
> > >>>
> > >>>
> > >>
> > >>
> > >>
>
---------------------------------------------------------------------
> > >> To unsubscribe, e-mail:
> java-user-unsubscribe@lucene.apache.org
> > >> For additional commands, e-mail:
> java-user-help@lucene.apache.org
> > >>
> > >>
> > >>
> > >
> > >
> > >
>
---------------------------------------------------------------------
> > > To unsubscribe, e-mail:
> java-user-unsubscribe@lucene.apache.org
> > > For additional commands, e-mail:
> java-user-help@lucene.apache.org
> > >
> > 
> > 
> >
>
---------------------------------------------------------------------
> > To unsubscribe, e-mail:
> java-user-unsubscribe@lucene.apache.org
> > For additional commands, e-mail:
> java-user-help@lucene.apache.org
> > 
> >
> 
>
---------------------------------------------------------------------
> To unsubscribe, e-mail:
> java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail:
> java-user-help@lucene.apache.org
> 
> 


---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org


Mime
View raw message