Return-Path: X-Original-To: archive-asf-public-internal@cust-asf2.ponee.io Delivered-To: archive-asf-public-internal@cust-asf2.ponee.io Received: from cust-asf.ponee.io (cust-asf.ponee.io [163.172.22.183]) by cust-asf2.ponee.io (Postfix) with ESMTP id 154B5200CFD for ; Wed, 6 Sep 2017 15:39:34 +0200 (CEST) Received: by cust-asf.ponee.io (Postfix) id 13EAF1609D9; Wed, 6 Sep 2017 13:39:34 +0000 (UTC) Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by cust-asf.ponee.io (Postfix) with SMTP id 33B501609D0 for ; Wed, 6 Sep 2017 15:39:33 +0200 (CEST) Received: (qmail 43716 invoked by uid 500); 6 Sep 2017 13:39:31 -0000 Mailing-List: contact user-help@predictionio.incubator.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@predictionio.incubator.apache.org Delivered-To: mailing list user@predictionio.incubator.apache.org Received: (qmail 43706 invoked by uid 99); 6 Sep 2017 13:39:31 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd3-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 06 Sep 2017 13:39:31 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd3-us-west.apache.org (ASF Mail Server at spamd3-us-west.apache.org) with ESMTP id 2F024183764 for ; Wed, 6 Sep 2017 13:39:31 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd3-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: 2.49 X-Spam-Level: ** X-Spam-Status: No, score=2.49 tagged_above=-999 required=6.31 tests=[DKIM_SIGNED=0.1, DKIM_VALID=-0.1, HTML_MESSAGE=2, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H3=-0.01, RCVD_IN_MSPIKE_WL=-0.01, RCVD_IN_SORBS_SPAM=0.5, T_KAM_HTML_FONT_INVALID=0.01] autolearn=disabled Authentication-Results: spamd3-us-west.apache.org (amavisd-new); dkim=pass (2048-bit key) header.d=occamsmachete-com.20150623.gappssmtp.com Received: from mx1-lw-us.apache.org ([10.40.0.8]) by localhost (spamd3-us-west.apache.org [10.40.0.10]) (amavisd-new, port 10024) with ESMTP id a_llJi4AJpdM for ; Wed, 6 Sep 2017 13:39:26 +0000 (UTC) Received: from mail-pg0-f53.google.com (mail-pg0-f53.google.com [74.125.83.53]) by mx1-lw-us.apache.org (ASF Mail Server at mx1-lw-us.apache.org) with ESMTPS id E567B5FD48 for ; Wed, 6 Sep 2017 13:39:25 +0000 (UTC) Received: by mail-pg0-f53.google.com with SMTP id 188so12037249pgb.2 for ; Wed, 06 Sep 2017 06:39:25 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=occamsmachete-com.20150623.gappssmtp.com; s=20150623; h=from:message-id:mime-version:subject:date:in-reply-to:cc:to :references; bh=b7zqNDbXbjJFbeT4wLmiavYsDqcRttrhqV2gfF3gtpw=; b=ZvMx/PxYs8/Ev1m1VxmU0Yjii80QZ2meuCk+VhnFTo0TNMHLa+i5+7xlZZZifC7iF+ ia0TYorRt/OtL92pJbFeR8+SiyaDfYuS0AvBDff7ibSDFAfctg4XHsVBgxU9Z6mEATuW rZmqq5kfE92iTpHgXDNn3b/2G5gKUAZSsqAu7yDgWkL7Z3G9gBFMmXgVr2XPf31upVfQ u1/lBrzTjei2gjf5F9iVhHv7NC+HzBFPg4xenCZoLbYIpa0mofn4C92kBnX+tTEiLcyf /zH6d+7kDmeIJxgSZJJlDDL6Pp2YjK8aez37E7pDTs2MNksnZioUcL9rs5qeC2FA25Dd 7Rfg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:message-id:mime-version:subject:date :in-reply-to:cc:to:references; bh=b7zqNDbXbjJFbeT4wLmiavYsDqcRttrhqV2gfF3gtpw=; b=To1yl5E4Guo2VyRrQfySUkY8MA3/wElUWpPSLVlf0/Lb9VChDlj/T2bIyaavUOyW4C 5yNmjeefYy4JkAGuSaExRhNOyNDy/wmUIclvyAzlayA2i0V1KZDFyqthRlP7RdDsi+a0 UedtT/4N4PnM9BfX97f3ROHiBqpc74bpYHy1e5S4LgEPyxOPbW/G94pOYnyO5WT9i0fg oo2YPUEZazb71we3/n8Q5CeE4SCZusMAwDjHP09vOhZG0Zjl5pKdG/xpcui2L0oVR92Q p78XhwBDVVvSO8kVqH/7AaBHCus7c9UZy23/IohHdJ7AqitZYNsho4RNuLlb8MgStmjj 5xnQ== X-Gm-Message-State: AHPjjUgjyCM47hNtym0qNkrtcefePV5BcQ4yX+tRLXD3SrkyoHu50CVZ 8T27iJJ80uy0FBdc X-Google-Smtp-Source: ADKCNb5ybjW/AReC3uDI7qch0MvMJQmBbp/7hI47z5rHIu/GSOw3Yp9lU7h/rPpXlTlY/8fzLpvdCA== X-Received: by 10.84.237.9 with SMTP id s9mr8256920plk.187.1504705159052; Wed, 06 Sep 2017 06:39:19 -0700 (PDT) Received: from [192.168.0.4] (c-24-18-213-211.hsd1.wa.comcast.net. [24.18.213.211]) by smtp.gmail.com with ESMTPSA id l131sm5931086pga.24.2017.09.06.06.39.17 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 06 Sep 2017 06:39:18 -0700 (PDT) From: Pat Ferrel Message-Id: Content-Type: multipart/alternative; boundary="Apple-Mail=_08005EF6-EC5A-441C-88CF-46A402B55B74" Mime-Version: 1.0 (Mac OS X Mail 10.3 \(3273\)) Subject: Re: Validate the built model Date: Wed, 6 Sep 2017 06:39:17 -0700 In-Reply-To: Cc: actionml-user@googlegroups.com, user@predictionio.incubator.apache.org To: Saarthak Chandra References: X-Mailer: Apple Mail (2.3273) archived-at: Wed, 06 Sep 2017 13:39:34 -0000 --Apple-Mail=_08005EF6-EC5A-441C-88CF-46A402B55B74 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=us-ascii We do cross-validation tests to see how well the model predicts actual = behavior. As to the best data mix, cross-validation works with any = engine tuning or data input. Typically this requires re-traiing between = test runs so make sure you use exatly the same training/test split. If = you want to examine the usefulness of different events you can compare = event-type 1 to event type 1 + event type 2 etc. This is made easier by = inputting all events, then using a test trick in the UR to mask out any = combination of events for the cross-validation, using the single = existing model so no need to re-train for this type of analysis. We have = an un-supported script that does this but I warn you that you are on = your own using it.=20 https://github.com/actionml/analysis-tools = On Sep 6, 2017, at 6:15 AM, Saarthak Chandra = wrote: Hi, With the Universal Recommender, 1. How can we validate the model after we train and deploy it? 2. How can we find an appropriate method of data mixing ?? Thanks --=20 Saarthak Chandra, Masters in Computer Science, Cornell University. --=20 You received this message because you are subscribed to the Google = Groups "actionml-user" group. To unsubscribe from this group and stop receiving emails from it, send = an email to actionml-user+unsubscribe@googlegroups.com = . To post to this group, send email to actionml-user@googlegroups.com = . To view this discussion on the web visit = https://groups.google.com/d/msgid/actionml-user/CAJHqc1rMSDD6w1WGxKkHqvVUG= Y9%2B3RfOOdtmUqY6C3Ew361TfA%40mail.gmail.com = . For more options, visit https://groups.google.com/d/optout = . --Apple-Mail=_08005EF6-EC5A-441C-88CF-46A402B55B74 Content-Transfer-Encoding: quoted-printable Content-Type: text/html; charset=us-ascii We do cross-validation tests to see how well the model = predicts actual behavior. As to the best data mix, cross-validation = works with any engine tuning or data input. Typically this requires = re-traiing between test runs so make sure you use exatly the same = training/test split. If you want to examine the usefulness of different = events you can compare event-type 1 to event type 1 + event type 2 etc. = This is made easier by inputting all events, then using a test trick in = the UR to mask out any combination of events for the cross-validation, = using the single existing model so no need to re-train for this type of = analysis. We have an un-supported script that does this but I warn you = that you are on your own using it. 

https://github.com/actionml/analysis-tools


On Sep 6, 2017, at 6:15 AM, Saarthak = Chandra <chandra.saarthak@gmail.com> wrote:

Hi,

With = the Universal Recommender,

1. How can we validate the model after we train = and deploy it?

2. = How can we = find an appropriate method of data mixing ??

Thanks
--
Saarthak Chandra,
Masters in Computer = Science,
Cornell University.

--
You received this message because you are subscribed to the Google = Groups "actionml-user" group.
To unsubscribe from this group and stop receiving emails from it, send = an email to actionml-user+unsubscribe@googlegroups.com.
To post to this group, send email to actionml-user@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/actionml-user/CAJHqc1rMSDD6w1= WGxKkHqvVUGY9%2B3RfOOdtmUqY6C3Ew361TfA%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

= --Apple-Mail=_08005EF6-EC5A-441C-88CF-46A402B55B74--