Mailing-List: contact user-help@mahout.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@mahout.apache.org
Received-SPF: pass (athena.apache.org: domain of srowen@gmail.com designates
 74.125.83.42 as permitted sender)
DomainKey-Signature: a=rsa-sha1; c=nofws;
        d=gmail.com; s=gamma;
        h=mime-version:in-reply-to:references:date:message-id:subject:from:to
         :content-type;
        b=Ql1ElqUJwNsBoJVvXDNHcm71mZZ6+nCR9VmbUfiviAp8ger1j89WLMfIf20qrV7kLS
         E4OEsZXWrY/PXL0BlD4sbs8IbiDUCBCI7kiTPt8TL1mIMmFkFYetBgEZmf+bHzlInH3F
         1zL8/7+GWVBYoEzcB93nQDpZAqSczhAg4Lo4w=
MIME-Version: 1.0
In-Reply-To: <BANLkTi=Tjd5qY-55v1FDrg_v0w0BA_LG1Q@mail.gmail.com>
References: <BANLkTi=Tjd5qY-55v1FDrg_v0w0BA_LG1Q@mail.gmail.com>
Date: Thu, 2 Jun 2011 08:15:10 +0100
Message-ID: <BANLkTikcqUpeocgtbNWb7TQAq=HV_KF+Gw@mail.gmail.com>
Subject: Re: PearsonCorrelationSimilarity returning NaN for user similarity
 with perfect match
From: Sean Owen <srowen@gmail.com>
To: user@mahout.apache.org
Content-Type: multipart/alternative; boundary=0016363b8f80cabee504a4b5657e

--0016363b8f80cabee504a4b5657e
Content-Type: text/plain; charset=UTF-8

I assume one or both has all the same ratings, at least in the overlapping
items. This means the standard deviation of their ratings is undefined, and
that's part of the formula. I think the answer is, that's just how it's
defined.

This tends to happen when the users have little overlap -- 1-2 items. And
ignoring that as a similarity is generally good.

But yes this is a reason you might not choose this metric.

On Thu, Jun 2, 2011 at 4:00 AM, Jason Smith <jasonparallel@gmail.com> wrote:

> What is the reasoning behind PearsonCorrelationSimilarity  returning
> NaN for userSimilarity when the two user's overlapping reviews match
> up perfectly?
> In my case of a limited set of rating values (1 to 5 stars) it seems
> quite possible that a user with a smaller number of ratings might have
> overlapping ratings with other users.  Am I missing something here.
>
>  // Note that sum of X and sum of Y don't appear here since they are
> assumed to be 0;
>    // the data is assumed to be centered.
>    double denominator = Math.sqrt(sumX2) * Math.sqrt(sumY2);
>    if (denominator == 0.0) {
>      // One or both parties has -all- the same ratings;
>      // can't really say much similarity under this measure
>      return Double.NaN;
>    }
>    return sumXY / denominator;
>

--0016363b8f80cabee504a4b5657e--