mahout-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Thomas Rewig <tre...@mufin.com>
Subject Re: Memory and Speed Questions for Item-Based-Recommender
Date Mon, 13 Jul 2009 13:48:07 GMT
Ok maybe I should talk in code to clarify my problem:

### Thats what I do: ###

    First I set the Item-Item-Matrix in a UserBased Way - here the 
Item-Item Correlations will be computed "on the fly"
   
       *** "Pre"-Recommender ***
        // set the model for the ItemItemMatrix
        this.similarityModel = new MySQLJDBCDataModel(cPoolDS, 
preferenceTable, userIDColumn, itemIDColumn, preferenceColumn);
       
        // set the "UserSimilarity" for the ItemItemMatrix
        this.similarityUserSimilarity = new 
EuclideanDistanceSimilarity(this.similarityModel);
       
        // set CaschingSimilarity
        this.cachingUserSimilarity = new 
CachingUserSimilarity(this.similarityUserSimilarity, this.similarityModel);
       
       *** Recommender ***
        // set the model for the Recommender
        this.model = new MySQLJDBCDataModel(cPoolDS, preferenceTable, 
userIDColumn, itemIDColumn, preferenceColumn);
      
        // set the userSimilarity for the Neighborhood
        this.userSimilarity = new PearsonCorrelationSimilarity( 
this.model );
       
        //set Neighborhood
        this.neighborhood = new NearestNUserNeighborhood( 
numNeighborhood, this.userSimilarity, this.model);
       
        // set the Recommender with the *cachingUserSimilarity*
        this.recommender = new GenericUserBasedRecommender(this.model, 
this.neighborhood, this.cachingUserSimilarity);

       --> I think I mixed here the UserSimilarity for the neighborhood 
in the Recommender (based at Users)  - and the UserSimilarity in the 
Pre-Recommender (based at Items) - and put both in the Recommender, the 
result is chaos in the recommendation, because the UserBasedRecommender 
doesn't work in that way

but
### What I want to do is: ###

        *** "Pre"-Recommender ***
        // set the model for the ItemItemMatrix
        this.similarityModel = new MySQLJDBCDataModel(cPoolDS, 
preferenceTable, userIDColumn, itemIDColumn, preferenceColumn);
       
        // set the "UserSimilarity" for the ItemItemMatrix
        this.similarityUserSimilarity = new 
EuclideanDistanceSimilarity(this.similarityModel);
       
        // set CaschingSimilarity
        this.cachingUserSimilarity = new 
CachingUserSimilarity(this.similarityUserSimilarity, this.similarityModel);
       
       *** Recommender ***
        // set the model for the Recommender
        this.model = new MySQLJDBCDataModel(cPoolDS, preferenceTable, 
userIDColumn, itemIDColumn, preferenceColumn);
      
        // Cast to itemSimilarity because the "Users" in the 
"Pre"-Recommender are Items
        this.cachingItemSimilarity = (CachingItemSimilarity) 
this.cachingUserSimilarity;
       
        // set the Recommender with the *cachingUserSimilarity*
        this.recommender = new GenericItemBasedRecommender(this.model, 
this.cachingItemSimilarity);

    -->I know, thats not working but in that way I could compute the 
ItemSimilarity on the fly (as a UserSimilarity) and must not precompute 
all Item-Item-Correlations, export them and import them as 
GenericItemSimilarity.ItemItemSimilarity. ( Ok with your new class I 
only have to compute and export them - the DB makes the rest)
I hope you understand my idea - also maybe there is a better way to do 
that, and I even don't see it, because I'm in the moment not completely 
familiar with the taste-code.


Sean Owen schrieb:
> I'm getting a little confused so let me clarify. You compute item-item
> similarities using a *user-based* recommender actually. I understand
> that part. In this first phase you treat items as users and item
> features as items. Makes sense. Sounds like that is working
That's working fine and the results / the correlations are plausible
> Then we arrive at the real recommendation engine. You have tried an
> item-based and user-based recommender. My first reaction is that you
> should probably be using an item-based recommender, or else the
> item-item similarities you computed are not used at all right?
>   
Yes that's what I want.
> Or did I misunderstand, and you are still talking about computing the
> item-item similarity with a user-based versus item-based recommender.
>
> I am not sure what you mean about using a user similarity metric in an
> item-based recommender. No, that algorithm does not use any notion of
> user similarity -- only item similarity. Which you precompute.
>
> Moving data to a JDBC- or Lucene-backed data store won't change the
> quality of recommendations, it will just change the memory and speed
> characteristics.
>   
Yes I know and that is important for practical use - at the moment all 
are only tests.
> On Mon, Jul 13, 2009 at 12:19 PM, Thomas Rewig<trewig@mufin.com> wrote:
>   
>> The UserBasedRecommender don't work for me! In comparative with the
>> Itembased-Recommender the Recommendations of the Userbase-Recommender are
>> really bad. Only/mostly popular Items will be recommended.
>> I think this is because I had a lot of Items and a lot of Users - but the
>> most Users occupied only a few Items and the overlap of Users is sparse. Now
>> there are a few other Users which occupied a lot of Items and randomly own a
>> Item of the User the recommendation is made for - so Items were be
>> recommended that have nothing in common with the profile of the user.
>>
>> So I have to use a recommender which bases only on the Item-Item-Matrix and
>> the Userprofile from the User the Recommendation is made for: The
>> ItemBasedRecommender. But I compute my Item-Item-Matrix with a
>> UserSimilarity. Is there a way to get a UserSimilarity in a
>> Item-Based-Recommender? Cast it somehow - so that I could compute the
>> Item-Item-Similarity "on the fly" like I do this now?
>>
>> Otherwise I would precompute the whole Item-Item-Matrix like I do it before
>> and put the Data with Sean's brand new MySQLJDBCItemSimilarity in the
>>  ItemBased - System. (I will test this now). Thank you for that!
>>     
>
>   

Mime
View raw message