Return-Path: X-Original-To: archive-asf-public-internal@cust-asf2.ponee.io Delivered-To: archive-asf-public-internal@cust-asf2.ponee.io Received: from cust-asf.ponee.io (cust-asf.ponee.io [163.172.22.183]) by cust-asf2.ponee.io (Postfix) with ESMTP id 8A539200D4E for ; Thu, 16 Nov 2017 16:59:57 +0100 (CET) Received: by cust-asf.ponee.io (Postfix) id 89454160BE5; Thu, 16 Nov 2017 15:59:57 +0000 (UTC) Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by cust-asf.ponee.io (Postfix) with SMTP id D89B6160BE6 for ; Thu, 16 Nov 2017 16:59:56 +0100 (CET) Received: (qmail 19696 invoked by uid 500); 16 Nov 2017 15:59:56 -0000 Mailing-List: contact user-help@predictionio.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@predictionio.apache.org Delivered-To: mailing list user@predictionio.apache.org Received: (qmail 19687 invoked by uid 99); 16 Nov 2017 15:59:56 -0000 Received: from mail-relay.apache.org (HELO mail-relay.apache.org) (140.211.11.15) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 16 Nov 2017 15:59:56 +0000 Received: from mail-yw0-f174.google.com (mail-yw0-f174.google.com [209.85.161.174]) by mail-relay.apache.org (ASF Mail Server at mail-relay.apache.org) with ESMTPSA id 61CCB1A02CD for ; Thu, 16 Nov 2017 15:59:55 +0000 (UTC) Received: by mail-yw0-f174.google.com with SMTP id g204so8129108ywa.6 for ; Thu, 16 Nov 2017 07:59:54 -0800 (PST) X-Gm-Message-State: AJaThX4rcM+F7AAGsbJOiHhhDFqySYYh+DUM+q48PKRMJVS57GTaI+Qv Uq5+UIIHaTEqsgQGBSCQgEeFz24wvSbq2iTf5RQ= X-Google-Smtp-Source: AGs4zMYvmsNbuP/jl2bBigrlG2ftyEYLJ3p+vD00oMayJItVHfh9goneQ8CwzewmKmMHBDvftvAMQg/Ilu4IsZ0vxts= X-Received: by 10.37.49.215 with SMTP id x206mr1164063ybx.169.1510847993450; Thu, 16 Nov 2017 07:59:53 -0800 (PST) MIME-Version: 1.0 Received: by 10.37.80.82 with HTTP; Thu, 16 Nov 2017 07:59:53 -0800 (PST) In-Reply-To: References: From: Suneel Marthi Date: Thu, 16 Nov 2017 15:59:53 +0000 X-Gmail-Original-Message-ID: Message-ID: Subject: Re: Log-likelihood based correlation test? To: user@predictionio.apache.org Cc: actionml-user Content-Type: multipart/alternative; boundary="001a11488ef2f599cc055e1bb58b" archived-at: Thu, 16 Nov 2017 15:59:57 -0000 --001a11488ef2f599cc055e1bb58b Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Indeed so. Ted Dunning is an Apache Mahout PMC and committer and the whole idea of Search-based Recommenders stems from his work and insights. If u didn't know, the PIO UR uses Apache Mahout under the hood and hence u see the LLR. On Thu, Nov 16, 2017 at 3:49 PM, Daniel Gabrieli wrote: > I am pretty sure the LLR stuff in UR is based off of this blog post and > associated paper: > > http://tdunning.blogspot.com/2008/03/surprise-and-coincidence.html > > Accurate Methods for the Statistics of Surprise and Coincidence > by Ted Dunning > > http://citeseerx.ist.psu.edu/viewdoc/summary?doi=3D10.1.1.14.5962 > > > On Thu, Nov 16, 2017 at 10:26 AM Noelia Os=C3=A9s Fern=C3=A1ndez < > noses@vicomtech.org> wrote: > >> Hi, >> >> I've been trying to understand how the UR algorithm works and I think I >> have a general idea. But I would like to have a *mathematical >> description* of the step in which the LLR comes into play. In the CCO >> presentations I have found it says: >> >> (PtP) compares column to column using >> *log-likelihood based correlation test* >> >> However, I have searched for "log-likelihood based correlation test" in >> google but no joy. All I get are explanations of the likelihood-ratio te= st >> to compare two models. >> >> I would very much appreciate a math explanation of log-likelihood based >> correlation test. Any pointers to papers or any other literature that >> explains this specifically are much appreciated. >> >> Best regards, >> Noelia >> > --001a11488ef2f599cc055e1bb58b Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
Indeed so. Ted Dunning is an Apache Mahout PMC and committ= er and the whole idea of Search-based Recommenders stems from his work and = insights.=C2=A0 If u didn't know, the PIO UR uses Apache Mahout under t= he hood and hence u see the LLR.

On Thu, Nov 16, 2017 at 3:49 PM, Daniel Gabrieli <dgabrieli@salesforce.com> wrote:
I am pretty sure the LLR stuff in UR i= s based off of this blog post and associated paper:

http://tdunning.blogspot.com/2008/03/surprise-= and-coincidence.html

Accurate Methods for= the Statistics of Surprise and Coincidence
by Ted Dunning
<= div>


On Thu, Nov 16, 2017 at 1= 0:26 AM Noelia Os=C3=A9s Fern=C3=A1ndez <noses@vicomtech.org> wrote:
Hi= ,

I've been trying to understand how the UR algorithm work= s and I think I have a general idea. But I would like to have a mathe= matical description of the step in which the LLR comes into play. I= n the CCO presentations I have found it says:

(PtP) compares c= olumn to column using log-likelihood based correlation test

<= br>
However, I have searched for "log-likelihood based correlatio= n test" in google but no joy. All I get are explanations of the likeli= hood-ratio test to compare two models.

I would very much appr= eciate a math explanation of log-likelihood based correlation test. Any poi= nters to papers or any other literature that explains this specifically are= much appreciated.

Best regards,
Noelia

--001a11488ef2f599cc055e1bb58b--