lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From dizh <d...@neusoft.com>
Subject Re: Re: how do I paginate Lucene search results deeply
Date Fri, 15 Mar 2013 01:38:52 GMT
Again:
searchAfter is good, but when you do Web paging, you will never know The user go to which
page.

Unlike google or yahoo... they only give you max 76 page as I see.

In my scenario, user can go to 10-20 or 20-30 and then again 10-20

by the way , the timestamp I stored is long, using: System.currentMillisecond() / 1000 , I
just get rid of Millisecond and the minmum is seconds

when I do paging , especially when I using sort ( the sorting field is just the timestamp)
, it is really eating memory

searchAfter(docId) require docId, I saw the lucene Core test code , there is a Test code using
searchAfter, but you still fisrtly get the docId

how do I get the docId? 4999990-5000000, I need to get 4888889, but if I really get 4888889
docId, there is no need to use searchAfter, you 

see 4888889 vs 5000000, just 20 , instead I think I will use search() to get all 5000000 and
then get the last ten.

My scenario is specail unlike most search engine requirements

Toke Eskildsen :
     (timestamp_in_ms << 10) | counter++ is a unique identifior ? 

Using timestamp_in_second can reduce terms but because the logs are too many , it doesn't
do much

According to Hibernate Search and mysql, I decide to use Shard by num.

for example:

1000000 logs as a index , and then create a new one, besides I can use this is I identifiy
the log using a timstamp one by one.

so : 
1000000 | 1000000 | 1000000 | 1000000

only one is active , which is doing indexing, and when reaching 1000000 , a new one created

when I search things, I combine them together, not using MultiReader, but using Concurrent
index searching just like ExexuteService.
And I must be careful of the Paging, because I need to caculate echo index contains how many
documents.

I don't whether this is good , but the is one way I thought of
---------------------------------------------------------------------------------------------------
Confidentiality Notice: The information contained in this e-mail and any accompanying attachment(s)

is intended only for the use of the intended recipient and may be confidential and/or privileged
of 
Neusoft Corporation, its subsidiaries and/or its affiliates. If any reader of this communication
is 
not the intended recipient, unauthorized use, forwarding, printing,  storing, disclosure or
copying 
is strictly prohibited, and may be unlawful.If you have received this communication in error,please

immediately notify the sender by return e-mail, and delete the original message and all copies
from 
your system. Thank you. 
---------------------------------------------------------------------------------------------------
Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message