mahout-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Otis Gospodnetic <otis_gospodne...@yahoo.com>
Subject Re: Shutting down RefreshHelper or Recommender
Date Tue, 14 Apr 2009 21:31:34 GMT

I'm moving this to mahout-user.

It looks like that Refresher background thread is still there:

$ (mvn clean compile war:war jetty:run) &
... the thole thing starts, I get recommendations for a user...
... fake file change by touching the file...
$ touch /rt/apps/recommendation-engine/re-input.txt
... wait a little...
Apr 14, 2009 5:28:53 PM org.slf4j.impl.JCLLoggerAdapter info
INFO: Reading file info...


Ha!
My understanding was that Sean removed this and this should no longer be happening on its
own, no?

Thanks,
Otis
--
Sematext -- http://sematext.com/ -- Lucene - Solr - Nutch



----- Original Message ----
> From: Otis Gospodnetic <otis_gospodnetic@yahoo.com>
> To: mahout-dev@lucene.apache.org
> Sent: Tuesday, April 14, 2009 4:58:46 PM
> Subject: Re: Shutting down RefreshHelper or Recommender
> 
> 
> Hi,
> 
> 
> > Should be even simpler:
> > 
> > Recommender r = ...;
> > ...
> > r.refresh();
> 
> Is that really so?  I don't see refresh() in any Recommenders in trunk:
> 
> otis@lesina:~/workspace/asf-mahout/core$ grep -ic refresh\(\) 
> ./src/main/java/org/apache/mahout/cf/taste/impl/recommender/*Recommender.java
> ./src/main/java/org/apache/mahout/cf/taste/impl/recommender/AbstractRecommender.java:0
> ./src/main/java/org/apache/mahout/cf/taste/impl/recommender/BooleanUserGenericUserBasedRecommender.java:0
> ./src/main/java/org/apache/mahout/cf/taste/impl/recommender/CachingRecommender.java:0
> ./src/main/java/org/apache/mahout/cf/taste/impl/recommender/GenericItemBasedRecommender.java:0
> ./src/main/java/org/apache/mahout/cf/taste/impl/recommender/GenericUserBasedRecommender.java:0
> ./src/main/java/org/apache/mahout/cf/taste/impl/recommender/ItemAverageRecommender.java:0
> ./src/main/java/org/apache/mahout/cf/taste/impl/recommender/ItemUserAverageRecommender.java:0
> ./src/main/java/org/apache/mahout/cf/taste/impl/recommender/TreeClusteringRecommender.java:0
> 
> 
> I do see the alternative method that takes in a Collection of Refreshables, as 
> in the code snippet I sent:
> 
> otis@lesina:~/workspace/asf-mahout/core$ grep -i refresh\(Coll 
> ./src/main/java/org/apache/mahout/cf/taste/impl/recommender/*Recommender.java
> ./src/main/java/org/apache/mahout/cf/taste/impl/recommender/BooleanUserGenericUserBasedRecommender.java:
 
> public void refresh(CollectionalreadyRefreshed) {
> ./src/main/java/org/apache/mahout/cf/taste/impl/recommender/CachingRecommender.java:
 
> public void refresh(CollectionalreadyRefreshed) {
> ./src/main/java/org/apache/mahout/cf/taste/impl/recommender/GenericItemBasedRecommender.java:
 
> public void refresh(CollectionalreadyRefreshed) {
> ./src/main/java/org/apache/mahout/cf/taste/impl/recommender/GenericUserBasedRecommender.java:
 
> public void refresh(CollectionalreadyRefreshed) {
> ./src/main/java/org/apache/mahout/cf/taste/impl/recommender/ItemAverageRecommender.java:
 
> public void refresh(CollectionalreadyRefreshed) {
> ./src/main/java/org/apache/mahout/cf/taste/impl/recommender/ItemUserAverageRecommender.java:
 
> public void refresh(CollectionalreadyRefreshed) {
> ./src/main/java/org/apache/mahout/cf/taste/impl/recommender/TreeClusteringRecommender.java:
 
> public void refresh(CollectionalreadyRefreshed) {
> 
> Is refresh() really in there somewhere?
> 
> > The RefreshHelper business is supposed to take care of the details.
> > Dependent components, all of them, will be refreshed first, in order, and
> > exactly once. Or am I missing something you want to do?
> 
> Well, that's one part, yes.  In particular, one thing that I want to do is 
> explicitly force the reload of the underlying input data (FileDataModel).
> 
> The other thing that I would like to be able to do is set new parameters for the 
> Recommender, or rather, classes that go into the recommender, such as the 
> neighbourhood size, minimal similarity, and a few parameters of my own that I 
> pass to my custom DataModel loader (it extends BooleanPrefUserFileDataModel and 
> takes several parameters in its ctor that tell it which users/items not to 
> load). I *think* that because of this and for this piece I still need to make a 
> completely new instance of the Recommender... because I don't want to just do a 
> refresh, I want to refresh, but with a different set of parameters.
> 
> > As to the other question, I don't know of any reason it would behave
> > differently on Java 5 versus 6. It is supposed to completely reload all
> > data.
> > 
> > I have a guess at the problem. I built in a 'feature' which lets you supply
> > 'update' files, named with a prefix like your main data file, which override
> > and update the main data file. The idea is it is a way to push updates to a
> > file without pushing the whole massive data file.
> 
> I like this idea.  As a matter of fact, I've thought along the same "diff lines" 
> while working on this recommender stuff.
> 
> > It is a good idea but the fact that other files in the same directory have
> > an effect too has already surprised even me - it created a test bug that a
> > kind user had to figure out for me. Is this possibly a factor? It might
> > explain why it behaves differently on a different machine.
> 
> I don't think that's it - I have only one "re-input.txt" file, which is my 
> (user,item,pref) file.  I create it elsewhere in the system and them just "mv" 
> it to the right place.  Before 0.1 Taste would see something changed and reload 
> the file (but, again, not on all systems) :)
> 
> I'll try explicit refresh now (with refresh(Collection
> hope will really reload the DataModel.
> 
> Otis
> 
> > Otherwise, hmm, would have to dig in with you and debug.
> > 
> > On Apr 14, 2009 4:20 PM, "Otis Gospodnetic" 
> > wrote:
> > 
> > 
> > OK, cool, so with refresh() this can become a push that looks like....:
> > 
> >    Recommender r
> >    ....
> >    public void init() {
> >      r = new SomeRecommender(...);
> >      DataModel dm = recommender.getDataModel();  // make DataModel
> > refreshable
> >      CollectionrefreshableDM = new ArrayList();
> >      refreshableDM.add(dm);
> >      r.refresh(refreshableDM);  // this seems to be needed according to
> > Refreshable javadoc?
> >    }
> > 
> >    // force DataModel refresh
> >    public void forceRefresh() {
> >      r.refresh(refreshableDM);
> >    }
> > ?
> > 
> > One more Q regarding refreshing:
> > Is this meant to completely reload the data (even if it has new users and
> > new items and new preferences)?  The reason I'm asking is because I'm seeing
> > a mixed behaviour depending on which system I run my recommender app on.  On
> > one system I can get the refresh to reload the data and "see" any new
> > users/items.  This system runs with Java 1.6 and the data lives on the local
> > disk.  On another system I can't get the changes to the data model file to
> > be visible to the recommender.  This system runs Java 1.5 and the data file
> > lives on the mounted NAS disk.  I *do* see "INFO: Reading file info...", but
> > the recommender still doesn't see, say, new users, and thus gives 0
> > recommendations for them.  Both systems run slightly pre-0.1 code.
> > 
> > Thanks, Otis ----- Original Message ---- > From: Sean Owen 
> > 
> > 
> > > To: mahout-dev@lucene.apache.org
> > 
> > > Sent: Tuesday, April 14, 2009 9:57:34 AM > Subject: Re: Shutting down
> > RefreshHelper or Recommender...
> > 
> > > Yes the change is in 0.1. No you can always have everything
> > reload/recompute > with the refresh() ...
> > 
> > > wrote: > > > Ah, I didn't even realize you took it out ... so it's
not in
> > 0.1? (I'm > behind on ...
> > > ----- Original Message ---- > From: Sean Owen > To:
> > 
> > > mahout-dev@lucene.apache.... > > > wrote: > > > > Hello,
> > > > Here is a
> > quick Taste thing that...


Mime
View raw message