mahout-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Sebastian Schelter <...@apache.org>
Subject Re: Can Mahout make recommendations while the recommender is being refreshed?
Date Fri, 19 Nov 2010 21:44:20 GMT
+dev

Good thing to talk about that issue, it's really not an ideal situation
to have FileDataModel need double memory at refresh time as it
effectively cuts in a half the amount of preferences that can be loaded
into per memory per machine.

Only two issues come to my mind if we would do a simple update of the data:

 a) existing preferences that are not included in the refreshed data
will not be removed
 b) recommenders reading the data at update time see an inconsistent
snapshot of the data

Let's assume that you only update and add preferences, then a) cannot
occur. I'm not sure how much of a problem b) would be in that case, you
could miss some cooccurrences when a preference is not added yet and
have some similarity value wrong if an update has not yet occurred. I'd
guess that is not a big problem as long as you don't cache the
inconsistent recommendations resulting from that.

So an idea could be to have a FileDataModel that never deletes
preferences and have that directly update the data. Any thoughts on this?

--sebastian

Am 19.11.2010 21:49, schrieb Sean Owen:
> ut the old dat


Mime
View raw message