Mailing-List: contact user-help@mahout.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@mahout.apache.org
Received-SPF: pass (athena.apache.org: domain of danquach@cs.ucla.edu
 designates 131.179.128.62 as permitted sender)
From: Daniel Quach <danquach@cs.ucla.edu>
Content-Type: text/plain; charset=us-ascii
Content-Transfer-Encoding: quoted-printable
Subject: Theory question about Pearson Correlation and user based recommender
Date: Wed, 9 May 2012 09:13:08 -0700
Message-Id: <78D22BF5-D91B-4804-BA8B-745E22A415FD@cs.ucla.edu>
To: user@mahout.apache.org
Mime-Version: 1.0 (Apple Message framework v1257)

I am running average absolute difference evaluations of a generic user =
based recommender that uses a threshold based neighborhood and pearson =
correlation to determine similarity.

I evaluated several recommenders for varying minimum thresholds for the =
neighborhood (0.9, 0.8, 0.7, 0.6, 0.5)

I noticed that as I decrease the threshold, the average absolute =
difference actually goes down, from:
0.85299 difference at 0.9 threshold of similarity
to
0.77667 difference at 0.5 threshold of similarity

My original intuition was that a higher threshold of similarity should =
result in more similar users appearing in each neighborhood, and hence =
should result in lower average absolute differences. However, this does =
not appear to be the case. Is there possibly some theoretical reason =
behind this? I repeated the same experiments using uncentered cosine =
similarity and those results reflect my original intuition (decreased =
difference when minimum thresholds for neighborhoods are higher)

I am performing experiments over the movie ratings from group lens.=