mahout-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From charlysf <charles.rue...@gmail.com>
Subject Item similarity very slow
Date Mon, 22 Jun 2009 20:06:05 GMT

Hello,

I would like to compute the item similarity for my data.

I have this table :

item_id, subject_id

An item is linked to a subject, which is a Taste, so I would like to have
the similarity between items, in fact, if they have the same subjects, or
not...

I tried to implement an AbstractJDBCDataModel for my database, and as I have
some boolean relationship between my item and my subject, I compute
similarities with TanimotoCoefficientSimilarity.

My recommender is GenericItemBasedRecommender and I use a
CachingRecommender.

In fact, do I have a better solution than :

for each item as item1
     give me the neighborhood(item1)


To retrieve the first neighborhood, I need around 20sec !

This is my log :

4306 [main] INFO
org.apache.mahout.cf.taste.impl.model.jdbc.AbstractJDBCDataModel - Executing
SQL query: SELECT COUNT(DISTINCT subject_id) FROM article_subject_linkage
4325 [main] INFO
org.apache.mahout.cf.taste.impl.model.jdbc.AbstractJDBCDataModel -
Retrieving all items...
4327 [main] INFO
org.apache.mahout.cf.taste.impl.model.jdbc.AbstractJDBCDataModel - Executing
SQL query: SELECT DISTINCT article_id FROM article_subject_linkage ORDER BY
article_id
4598 [main] INFO
org.apache.mahout.cf.taste.impl.model.jdbc.AbstractJDBCDataModel -
Retrieving item ID '25'
4618 [main] INFO
org.apache.mahout.cf.taste.impl.model.jdbc.AbstractJDBCDataModel - Executing
SQL query: SELECT 1 FROM article_subject_linkage WHERE article_id=?
4621 [main] INFO
org.apache.mahout.cf.taste.impl.model.jdbc.AbstractJDBCDataModel -
Retrieving item ID '25'
4622 [main] INFO
org.apache.mahout.cf.taste.impl.model.jdbc.AbstractJDBCDataModel - Executing
SQL query: SELECT 1 FROM article_subject_linkage WHERE article_id=?
4624 [main] INFO
org.apache.mahout.cf.taste.impl.model.jdbc.AbstractJDBCDataModel -
Retrieving number of items in model
4624 [main] INFO
org.apache.mahout.cf.taste.impl.model.jdbc.AbstractJDBCDataModel - Executing
SQL query: SELECT COUNT(DISTINCT article_id) FROM article_subject_linkage
4642 [main] INFO
org.apache.mahout.cf.taste.impl.model.jdbc.AbstractJDBCDataModel -
Retrieving all items...
4642 [main] INFO
org.apache.mahout.cf.taste.impl.model.jdbc.AbstractJDBCDataModel - Executing
SQL query: SELECT DISTINCT article_id FROM article_subject_linkage ORDER BY
article_id
4772 [main] INFO
org.apache.mahout.cf.taste.impl.model.jdbc.AbstractJDBCDataModel -
Retrieving number of user preferring items in model 25
4777 [main] INFO
org.apache.mahout.cf.taste.impl.model.jdbc.AbstractJDBCDataModel - Executing
SQL query: SELECT COUNT(1) FROM article_subject_linkage tp1 INNER JOIN
article_subject_linkage tp2 ON (tp1.subject_id=tp2.subject_id) WHERE
tp1.article_id=? and tp2.article_id=?
4778 [main] INFO
org.apache.mahout.cf.taste.impl.model.jdbc.AbstractJDBCDataModel -
Retrieving number of user preferring item in model 25
4778 [main] INFO
org.apache.mahout.cf.taste.impl.model.jdbc.AbstractJDBCDataModel - Executing
SQL query: SELECT COUNT(1) FROM article_subject_linkage WHERE article_id=?
4779 [main] INFO
org.apache.mahout.cf.taste.impl.model.jdbc.AbstractJDBCDataModel -
Retrieving number of user preferring item in model 3220
4779 [main] INFO
org.apache.mahout.cf.taste.impl.model.jdbc.AbstractJDBCDataModel - Executing
SQL query: SELECT COUNT(1) FROM article_subject_linkage WHERE article_id=?
4781 [main] INFO
org.apache.mahout.cf.taste.impl.model.jdbc.AbstractJDBCDataModel -
Retrieving number of user preferring items in model 25
4782 [main] INFO
org.apache.mahout.cf.taste.impl.model.jdbc.AbstractJDBCDataModel - Executing
SQL query: SELECT COUNT(1) FROM article_subject_linkage tp1 INNER JOIN
article_subject_linkage tp2 ON (tp1.subject_id=tp2.subject_id) WHERE
tp1.article_id=? and tp2.article_id=?
4785 [main] INFO
org.apache.mahout.cf.taste.impl.model.jdbc.AbstractJDBCDataModel -
Retrieving number of user preferring item in model 25
4785 [main] INFO
org.apache.mahout.cf.taste.impl.model.jdbc.AbstractJDBCDataModel - Executing
SQL query: SELECT COUNT(1) FROM article_subject_linkage WHERE article_id=?
4786 [main] INFO
org.apache.mahout.cf.taste.impl.model.jdbc.AbstractJDBCDataModel -
Retrieving number of user preferring item in model 3221
4787 [main] INFO
org.apache.mahout.cf.taste.impl.model.jdbc.AbstractJDBCDataModel - Executing
SQL query: SELECT COUNT(1) FROM article_subject_linkage WHERE article_id=?
4788 [main] INFO
org.apache.mahout.cf.taste.impl.model.jdbc.AbstractJDBCDataModel -
Retrieving number of user preferring items in model 25
4789 [main] INFO
org.apache.mahout.cf.taste.impl.model.jdbc.AbstractJDBCDataModel - Executing
SQL query: SELECT COUNT(1) FROM article_subject_linkage tp1 INNER JOIN
article_subject_linkage tp2 ON (tp1.subject_id=tp2.subject_id) WHERE
tp1.article_id=? and tp2.article_id=?
4790 [main] INFO
org.apache.mahout.cf.taste.impl.model.jdbc.AbstractJDBCDataModel -
Retrieving number of user preferring item in model 25
4790 [main] INFO
org.apache.mahout.cf.taste.impl.model.jdbc.AbstractJDBCDataModel - Executing
SQL query: SELECT COUNT(1) FROM article_subject_linkage WHERE article_id=?
4791 [main] INFO
org.apache.mahout.cf.taste.impl.model.jdbc.AbstractJDBCDataModel -
Retrieving number of user preferring item in model 3222
4791 [main] INFO
org.apache.mahout.cf.taste.impl.model.jdbc.AbstractJDBCDataModel - Executing
SQL query: SELECT COUNT(1) FROM article_subject_linkage WHERE article_id=?
4792 [main] INFO
org.apache.mahout.cf.taste.impl.model.jdbc.AbstractJDBCDataModel -
Retrieving number of user preferring items in model 25
4792 [main] INFO
org.apache.mahout.cf.taste.impl.model.jdbc.AbstractJDBCDataModel - Executing
SQL query: SELECT COUNT(1) FROM article_subject_linkage tp1 INNER JOIN
article_subject_linkage tp2 ON (tp1.subject_id=tp2.subject_id) WHERE
tp1.article_id=? and tp2.article_id=?
4793 [main] INFO
org.apache.mahout.cf.taste.impl.model.jdbc.AbstractJDBCDataModel -
Retrieving number of user preferring item in model 25
4793 [main] INFO
org.apache.mahout.cf.taste.impl.model.jdbc.AbstractJDBCDataModel - Executing
SQL query: SELECT COUNT(1) FROM article_subject_linkage WHERE article_id=?
4793 [main] INFO
org.apache.mahout.cf.taste.impl.model.jdbc.AbstractJDBCDataModel -
Retrieving number of user preferring item in model 3223
4794 [main] INFO
org.apache.mahout.cf.taste.impl.model.jdbc.AbstractJDBCDataModel - Executing
SQL query: SELECT COUNT(1) FROM article_subject_linkage WHERE article_id=?
4794 [main] INFO
org.apache.mahout.cf.taste.impl.model.jdbc.AbstractJDBCDataModel -
Retrieving number of user preferring items in model 25
4794 [main] INFO
org.apache.mahout.cf.taste.impl.model.jdbc.AbstractJDBCDataModel - Executing
SQL query: SELECT COUNT(1) FROM article_subject_linkage tp1 INNER JOIN
article_subject_linkage tp2 ON (tp1.subject_id=tp2.subject_id) WHERE
tp1.article_id=? and tp2.article_id=?
4795 [main] INFO
org.apache.mahout.cf.taste.impl.model.jdbc.AbstractJDBCDataModel -
Retrieving number of user preferring item in model 25
4796 [main] INFO
org.apache.mahout.cf.taste.impl.model.jdbc.AbstractJDBCDataModel - Executing
SQL query: SELECT COUNT(1) FROM article_subject_linkage WHERE article_id=?
4796 [main] INFO
org.apache.mahout.cf.taste.impl.model.jdbc.AbstractJDBCDataModel -
Retrieving number of user preferring item in model 3224
4797 [main] INFO
org.apache.mahout.cf.taste.impl.model.jdbc.AbstractJDBCDataModel - Executing
SQL query: SELECT COUNT(1) FROM article_subject_linkage WHERE article_id=?


I don't understand why mahout makes very often the same query, and don't
store the result, for example in my case, every time, it needs again the
"Retrieving number of user preferring item in model 25"

Thank you very much for your help
-- 
View this message in context: http://www.nabble.com/Item-similarity-very-slow-tp24154435p24154435.html
Sent from the Mahout User List mailing list archive at Nabble.com.


Mime
View raw message