Mailing-List: contact user-help@mahout.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@mahout.apache.org
Received-SPF: pass (nike.apache.org: domain of florent.empis@gmail.com
 designates 209.85.214.42 as permitted sender)
DomainKey-Signature: a=rsa-sha1; c=nofws;
        d=gmail.com; s=gamma;
        h=mime-version:in-reply-to:references:date:message-id:subject:from:to
         :content-type;
        b=KxyS3xfb3YS0x45JJhqlqrWivPeQJGPtSuspHC+u9UY3ctcxMo6myz+B8YMyJ5CnYY
         nYFlJi/fDRh88CiXwgZsC0C5kJx6gki3j9cJmXkhPu2MUDFNgIocSoZ8Xi9klE6twmjs
         sy7fqr21cmBzvbZq2FcmXumzC4L3pLMmRhUsk=
MIME-Version: 1.0
In-Reply-To: <AANLkTi=aDWQTEf4mB8orCiHxTjPpDUJyXcts8LHJST_s@mail.gmail.com>
References: <624845ab.e2ad.12a1f6abdf6.Coremail.woshidustin@126.com>
	<AANLkTi=aDWQTEf4mB8orCiHxTjPpDUJyXcts8LHJST_s@mail.gmail.com>
Date: Fri, 30 Jul 2010 16:30:04 +0200
Message-ID: <AANLkTimLmWRhJgPvrP=UXzj_xczSOy+_3TJ8c6y9CXXq@mail.gmail.com>
Subject: Re: Could you improve the AbstractJDBCDataModel?
From: Florent Empis <florent.empis@gmail.com>
To: user@mahout.apache.org
Content-Type: multipart/alternative; boundary=00151750e552ce3686048c9baf86

--00151750e552ce3686048c9baf86
Content-Type: text/plain; charset=ISO-8859-1

Actually, I discussed a while back the creation of a caching strategy for
the datamodel, through Memcache.
I did begin an implementation, but other stuff (namely, a new job) got in
the way....

Another option would be to go for a library such as Hibernate or Ibatis and
throw efficient caching (such as EhCache) at it.
You'd maybe loose some speed (particularly with the added abstraction layer
of Hibernate), but it would probably not be too big of job to implement
(particularly with Ibatis, where it would be a matter a few days: take the
existing statements, convert them to plain sql request, write caching
configuration and you're all set)


2010/7/29 Sean Owen <srowen@gmail.com>

> That's rather the point of the JDBC-backed models -- they're for the
> case where you can't load data in memory. If you can, then load them
> in memory.
>
> Yes it's not fast to use JDBC. It only really makes sense with
> algorithms that, through their nature or caching, don't read a lot of
> data. You want to use the Caching* wrappers everywhere.
>
> You might write some kind of DataModel wrapper which caches some
> number of user preferences to mitigate this effect, yes.
>
> 2010/7/29 Young <woshidustin@126.com>:
> > Hi All,
> > I use MySQLJDBCDataModel for ItemBasedRecommender for online
> recommendation. I have put other stuff in the memory and do the
> itemsimilarity matrix precomputation.
> >
> > I find when I do one single recommendation, the
> getPreferencesFromUser(long id) method (in )will be called thousands of
> times. I think it is okay when using in-memory datamodel. But if it is in
> the AbstractJDBCDataModel, it will take much more time because the *same*
> mysql queries will be excuted thousands of times.
> >
> > Thanks.
> > ---Young
>

--00151750e552ce3686048c9baf86--