mahout-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Sean Owen <sro...@gmail.com>
Subject Re: Changing in-memory DataModel to a DB dependent only DataModel after building recommender
Date Mon, 21 Jan 2013 20:59:46 GMT
That's all right, but it's not something implemented in this project,
except maybe in SVDRecommender to some extent.

On Mon, Jan 21, 2013 at 8:57 PM, Osman Başkaya
<osman.baskaya@computer.org> wrote:
> Hey Sean,
>
> Once you compute user and item factors, you don't need to create a dense
> matrix in which all the user's predicted ratings contains. Of course, one
> dependency remains: you may want to know which user rates which movies.
> Otherwise you recommend movies which are rated already. However, computing
> all predictions and storing them does not provides this, neither.
>
> On Mon, Jan 21, 2013 at 10:40 PM, Sean Owen <srowen@gmail.com> wrote:
>
>> If you don't have the data in memory you can't compute anything. The
>> recommender itself doesn't do anything without data. That's why it
>> seemed like you really just wanted to compute everything offline
>> first, in which case the simplest solution is to store it however you
>> like and fetch that result however you like.
>>
>> On Mon, Jan 21, 2013 at 8:22 PM, Ceyhun Can ÜLKER <ceyhuncanu@gmail.com>
>> wrote:
>> > Hi again,
>> >
>> > Thank you for your quick reply, Sean. I couldn't understand one point.
>> What
>> > do you mean by pre-compute and store recommendations? Doesn't it mean
>> > having a dense (rather filled?) rating matrix? So it would make memory
>> > usage much worse, even if it is possible. Wouldn't it better to keep the
>> > model and compute whenever necessary?
>> >
>> > Thanks
>> > Ceyhun Can Ulker
>> >
>> >
>> > On Mon, Jan 21, 2013 at 9:58 PM, Sean Owen <srowen@gmail.com> wrote:
>> >
>> >> You would have to write this yourself, yes.
>> >> If you're not keeping the data in memory, you're not updating the
>> >> results in real-time. So there's no real need to keep any DataModel
>> >> around at all. Just pre-compute and store recommendations and update
>> >> them periodically. Nothing has to be on-line then.
>> >>
>> >> On Mon, Jan 21, 2013 at 7:54 PM, Ceyhun Can ÜLKER <ceyhuncanu@gmail.com
>> >
>> >> wrote:
>> >> > Hello,
>> >> >
>> >> > In our application we are using ReloadFromJDBCDataModel for its speed
>> >> > advantage of in-memory representation and being able to update
>> >> periodically
>> >> > to pull in new data from a database source.
>> >> >
>> >> > However, once the recommender is build we do not want to keep the
>> ratings
>> >> > data in memory (we would like to query the database when rating data
>> is
>> >> > needed). We want to replace the ReloadFromJDBCDataModel with a
>> >> > MySqlJDBCDataModel after build. But there is no setter method for it,
>> >> > furthermore, the field that keeps the DataModel is in
>> AbstractRecommender
>> >> > (superclass of SVDRecommender) and it is declared final.
>> >> >
>> >> > We thought we could write a new class that derives from DataModel,
>> which
>> >> > initial keeps a Reload model instance (let's call this delegateModel),
>> >> has
>> >> > a setter method for it, and delegates all DataModel methods, so that
>> we
>> >> > could set this delegateModel field to another instance, say
>> >> > MySqlJDBCDataModel instance. Is this a good method for removing
>> in-memory
>> >> > representation dependency after the build process?
>> >> >
>> >> > How can we achieve this change? Or is there an alternative and better
>> way
>> >> > to achieve this?
>> >> >
>> >> > Thanks
>> >> > Ceyhun Can Ulker
>> >>
>>
>
>
>
> --
> Osman Başkaya
> Koc University
> MS Student | Computer Science and Engineering

Mime
View raw message