Mailing-List: contact user-help@mahout.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@mahout.apache.org
Received-SPF: pass (nike.apache.org: domain of manuel.blechschmidt@gmx.de
 designates 212.227.15.19 as permitted sender)
Content-Type: text/plain; charset=iso-8859-1
Mime-Version: 1.0 (Apple Message framework v1283)
Subject: Re: Which database should I use with Mahout
From: Manuel Blechschmidt <Manuel.Blechschmidt@gmx.de>
In-Reply-To: 
 <CAEccTyyJxp3i=CMuUzUOFiOTN3OiZzC0mmec--2+wJMSgK4uhA@mail.gmail.com>
Date: Sun, 19 May 2013 20:01:37 +0200
Content-Transfer-Encoding: quoted-printable
Message-Id: <6F9272F7-D39A-4D3D-A3B1-5F590567E5BF@gmx.de>
References: <1368982327.55670.YahooMailNeo@web140005.mail.bf1.yahoo.com>
 <CAEccTyzhauKtaP1OxO1AP2Ev_BaY35CuCoLk-SnoZNF2M9R3Zw@mail.gmail.com>
 <CAH9ofMa6+vYjm3vGvMOKrEbOYR9LsNZ3e-JD7+-PBtdbKRvTTA@mail.gmail.com>
 <CAEccTyyJxp3i=CMuUzUOFiOTN3OiZzC0mmec--2+wJMSgK4uhA@mail.gmail.com>
To: user@mahout.apache.org

Hi Tevfik,
one request to the recommender could become more then 1000 queries to =
the database depending on which recommender you use and the amount of =
preferences for the given user.

The problem is not if you are using SQL, NoSQL, or any other query =
language. The problem is the latency of the answers.

An average tcp package in the same data center takes 500 =B5s. A main =
memory reference 0,1 =B5s. This means that your main memory of your java =
process can be accessed 5000 times faster then any other process like a =
database connected via TCP/IP.

http://www.eecs.berkeley.edu/~rcs/research/interactive_latency.html

Here you can see a screenshot that shows that database communication is =
by far (99%) the slowest component of a recommender request:

https://source.apaxo.de/MahoutDatabaseLowPerformance.png

If you do not want to cache your data in your Java process you can use a =
complete in memory database technology like SAP HANA =
http://www.saphana.com/welcome or EXASOL http://www.exasol.com/

Nevertheless if you are using these you do not need Mahout anymore.

An architecture of a Mahout system can be seen here:
=
https://github.com/ManuelB/facebook-recommender-demo/blob/master/docs/Reco=
mmenderArchitecture.png

Hope that helps
    Manuel

Am 19.05.2013 um 19:20 schrieb Sean Owen:

> I'm first saying that you really don't want to use the database as a
> data model directly. It is far too slow.
> Instead you want to use a data model implementation that reads all of
> the data, once, serially, into memory. And in that case, it makes no
> difference where the data is being read from, because it is read just
> once, serially. A file is just as fine as a fancy database. In fact
> it's probably easier and faster.
>=20
> On Sun, May 19, 2013 at 10:14 AM, Tevfik Aytekin
> <tevfik.aytekin@gmail.com> wrote:
>> Thanks Sean, but I could not get your answer. Can you please explain =
it again?
>>=20
>>=20
>> On Sun, May 19, 2013 at 8:00 PM, Sean Owen <srowen@gmail.com> wrote:
>>> It doesn't matter, in the sense that it is never going to be fast
>>> enough for real-time at any reasonable scale if actually run off a
>>> database directly. One operation results in thousands of queries. =
It's
>>> going to read data into memory anyway and cache it there. So, =
whatever
>>> is easiest for you. The simplest solution is a file.
>>>=20
>>> On Sun, May 19, 2013 at 9:52 AM, Ahmet Ylmaz
>>> <ahmetyilmazefendi@yahoo.com> wrote:
>>>> Hi,
>>>> I would like to use Mahout to make recommendations on my web site. =
Since the data is going to be big, hopefully, I plan to use hadoop =
implementations of the recommender algorithms.
>>>>=20
>>>> I'm currently storing the data in mysql. Should I continue with it =
or should I switch to a nosql database such as mongodb or something =
else?
>>>>=20
>>>> Thanks
>>>> Ahmet

--=20
Manuel Blechschmidt
M.Sc. IT Systems Engineering
Dortustr. 57
14467 Potsdam
Mobil: 0173/6322621
Twitter: http://twitter.com/Manuel_B