mahout-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Chen_1st <...@live.cn>
Subject Re: Two learning competitions that might be of interest for Mahout
Date Tue, 15 Feb 2011 18:34:35 GMT
Hi, Sean,

Sorry for my poor English.

>>Hmm, not sure I understand. No, it's not true that real-life data
>>regularly omits the user's top ratings. Why would that be?

In reallife applications, it's impossible for users to provide ratings for
all their favoriate tracks, right? It's the same effect as omitting some top
rated tracks.

>>How would you score the recommendations by holding out a random
>>subset? That subset is definitely *not* representative of good
>>recommendations -- you might be picking out things the user hates.

Consider the example: the top favoriate tracks of the user are complete_set
= {1, 2, ..., 10}, and user only provide ratings on randomly_selected_subset
= {1, 2, ..., 5}, here we assume the user randomly selected 5 tracks from
the complete_set and rated them. Let the recommender system predict top 5
tracks for the user, if it can correctly hit 3 in randomly_selected_subset,
it's with high probability better than hit only 1,

The above is the illustration how to  apply recall@5. Precision and NDCG are
similar.
2011/2/16 Sean Owen <srowen@gmail.com>

> Hmm, not sure I understand. No, it's not true that real-life data
> regularly omits the user's top ratings. Why would that be?
>
> How would you score the recommendations by holding out a random
> subset? That subset is definitely *not* representative of good
> recommendations -- you might be picking out things the user hates.
>
> Precision / recall don't really make sense unless you think you're
> holding out "good" recommendations and those would have to be top
> rated items.
>
> Sean
>
> On Tue, Feb 15, 2011 at 5:36 PM, Chen_1st <y.c@live.cn> wrote:
> > Hi, Sean,
> >
> > I cannot agree with you.
> >
> > The small problem you mentioned might incur difficulties in prediction
> > indeed, but such problem also occurs in real life applications, right?
> >
> > As to the big problem you mentioned,  of course, we don't have the
> complete
> > set of true result, but if the available subset of true result is
> randomly
> > selected from the complete set, I think the evaluation criteria like
> > recall@k, precision@k, or ndcg are still meaningful.
> >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message