predictionio-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Gustavo Frederico <gustavo.freder...@thinkwrap.com>
Subject Re: How to score the model? (lift)
Date Tue, 27 Sep 2016 15:00:28 GMT
I know of at least one good A/B testing service that uses JavaScript
at the Front End. So that would work and it wouldn't not touch the
PIO+UR stack. But your answer made me think of something else: Is
there a way to score two models with PIO?  Maybe Google Analytics is
still preferable even for that task...

Thanks
Gustavo

On Mon, Sep 26, 2016 at 7:21 PM, Pat Ferrel <pat@occamsmachete.com> wrote:
> A/B testing it the typical way to compare the performance of 2 or more versions of an
app. You should make sure that the versions only vary in 1 way to isolate what you are comparing.
So channel some traffic to a page “without recommendations” and some amount of traffic
to a page “with recommendations” and measure your conversion KPI for both cases. If your
conversion is a sale or purchase you may have to split all users permanently over the test
period. if you are only measuring CTR you may be able to split impressions since you don’t
have to track users as a group all the way to a purchase.
>
> On Sep 26, 2016, at 5:45 AM, Gustavo Frederico <gustavo.frederico@thinkwrap.com>
wrote:
>
> Allow me to elaborate some more: there's a second question related to
> the original one. How to score a model in Production?
>
> Thanks
>
> On Fri, Sep 23, 2016 at 2:58 PM, Gustavo Frederico
> <gustavo.frederico@thinkwrap.com> wrote:
>> Considering PIO + the UR template, what are some of the ways to score
>> the model? I can think of some A/B testing but that would imply
>> relying on things outside the PIO + UR stack. Is there a way to
>> measure lift or "score the model"? Cross-validation with separate
>> testing set?
>>
>> Thanks
>>
>> Gustavo
>

Mime
View raw message