lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Vladimir Olenin" <VOle...@cihi.ca>
Subject RE: Help on search
Date Tue, 07 Nov 2006 18:58:13 GMT
You might actually try to look for some 'names database' (similar to Wordnet). Someone has
probably already compiled a list of english 'names' and their common short forms (eg, 'Vlad'
for 'Vladimir', 'Fred' for 'Frederich', etc). Alternatevly, compile such DB yourself (and
don't forget to publish it under OSS license afterwards!!!! :) ) - if there is none right
now it really can be useful for quite a few applications.

Once you have the DB your can do similar things as with 'english synomym' DB (not sure what
Lucene notations are for this) - lookup 'synonym' for the name and thus be able to search
all forms.... Or use such DB for custom stemming, as Erick suggested.

Vlad

-----Original Message-----
From: Erick Erickson [mailto:erickerickson@gmail.com] 
Sent: Tuesday, November 07, 2006 1:27 PM
To: java-user@lucene.apache.org
Subject: Re: Help on search

<<<How can I make a rule for stemming names?>>>

That's the problem all right. And I don't have any good answer. Wildcarding could work for
you, but it's really the same question, isn't it? "How do you transform a typed name into
a wildcard query?" is, I think, equivalent to your question above (assuming that wildcarding
is really truncation. i.e.
Fred* and not Fred*i*). And if you can answer it, you can stem or wildcard......

Could a really simple rule like "only use the first 4 letters" work? Of course there are obvious
names for which this wouldn't work. But is something like that "good enough"?

There is one other possibility, soundex/metaphone. There's a discussion in Lucene in Action
about how to do this. The basic idea is that you index and search on words that "sound like"
what's in the document. Whether "sounds like" makes Fred and Frederich equivalent is the unknown
here.....

Not much help I know

Good luck
Erick

On 11/7/06, Alice <alicelista@gmail.com> wrote:
>
> Thank you for answering.
>
> I thought about stemming at index and search time, but Im dealing with 
> names.
> How can I make a rule for stemming names?
>
> I'm thinking about different names, I thought I could use Lucene to 
> help users find whomever they're looking for without having to spell 
> their friends names correctly or fully.
>
> That's why I gave the example about 'Frederich', it is very common 
> that users enter 'Fred'.
>
> I'm going to read about the wildcards.
>
> Anymore suggestions are very welcome.
>
> Thanks
>
> -----Original Message-----
> From: Erick Erickson [mailto:erickerickson@gmail.com]
> Sent: terça-feira, 7 de novembro de 2006 15:12
> To: java-user@lucene.apache.org
> Subject: Re: Help on search
>
> You really have to get away from thinking about lucene like you would 
> a database, or you'll have endless problems <G>.
>
> If you want just a bag of name parts, why use separate fields? For 
> instance, index both firstname and lastname in a name field in each 
> document. Then all you have to do to find all documents that have 
> Brown in them is search on the single field name. There's no need to 
> construct complex or clauses, which is what I suspect you're doing 
> now.....
>
> You can also index firstname, lastname, and a third field "bagofnames"
> that
> contains both if you want.
>
> Vladimir is right, unless you take some actions you always get exact 
> matches when you search, so I'm not surprised that searching for Fred 
> does NOT return Frederich-containing documents.
>
> Whether wildcarding is your best strategy is debatable, it depends 
> upon your intent. You could consider stemming both at index and search 
> time.
> Generically, all that means is that you index "Fred" for "Frederich". 
> and at search time you search for "Fred". There are stemming analyzers 
> in Lucene, but whether they do what you want only you can decide. Be 
> aware that wildcard searches frequently run afoul of a TooManyClauses 
> exception.
> There
> is a bunch of information from people far more knowledgeable than me 
> in this mail archive if you search on "wildcard". It's a more 
> complicated matter than you might think at first.
>
> Note that the TooManyClauses exception very likely won't show up in 
> your test data sets. It won't show up until you put in a bunch of 
> data. See TooManyClauses in the javadoc.....
>
> I'd argue if you can create rules for wildcarding, you can create 
> rules for stemming By that I mean that if you have a programmatic way 
> to turn "Frederich" into "Fred", you can just index and search on 
> "Fred" equally easily and not have to deal with wildcard complexity.
>
> Of course, since I don't know what problem you're actually trying to 
> solve, this may be irrelevant....
>
> Best
> Erick
>
> On 11/7/06, Alice <alicelista@gmail.com> wrote:
> >
> > Hello!
> >
> > I am totally new to Lucene and I'm trying to use it with my web 
> > application.
> >
> >
> >
> > What I'm doing is reading a table from my database line by line and 
> > indexing the columns.
> >
> > I read the users data such as First Name, Last Name, Email and so on.
> >
> >
> >
> > I hava a field by column, such as: firstName="Frederich"
> lastName="Brown"
> >
> > My intetion is to make users find other users using name, lastName 
> > or email.
> >
> > What is the best way to do it?
> >
> > If the user enter "Fred Brown", as I use queryParser, this string is 
> > broken to "Fred" "Brown" and I search those tokens in every field.
> >
> >
> >
> > As "Brown" was indexed, the user's registry is found.
> >
> > But if the user enters "Fred", no user is found.
> >
> > Why is that? I thought it would return the user "Frederich Brown"
> either.
> >
> > Can someone help?
> >
> >
> >
> > Thanks!
> >
> >
> >
> >
> >
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
>
>

---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org


Mime
View raw message