predictionio-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Pat Ferrel <...@occamsmachete.com>
Subject Re: Incremental model training in real time event processing
Date Thu, 01 Dec 2016 17:37:38 GMT
No, lambda has nothing to do with how much data is used in training. Lambda just say that there
is a batch/background process that account for changes in data in non-real-time. It is theoretically
possible but not supported yet. However you may be training too often...

The Universal Recommender uses the entire accumulated dataset to train but uses real-time
user history data to make queries for user-based recommendations. The training rule-of-thumb
is to do it as often as your catalog of items changes. By training daily you are saying that
you have new items every day? 

Also there is no reason to re-deploy. If the PredictionServer is deployed the model will be
hot-swapped into it at the end of every successful `pio train`.


On Nov 29, 2016, at 1:21 AM, Saurav Sarkar <saurav.sarkar1@gmail.com> wrote:

Hi All,

We are using universal recommender for our use case.

We have both batch processing and event processing scenarios.

In this context i have the following question.

1. We have a batch processing setup, import the events in batch every day, train and deploy.

2. We also send real time events, train the model manually and deploy.

Can the second step be done by incremental training on the already existing model.
As per the lambda architecture setup it should be possible,

Best Regards,
Saurav


Mime
View raw message