Mailing-List: contact user-help@mahout.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@mahout.apache.org
Received-SPF: pass (athena.apache.org: domain of goksron@gmail.com designates
 209.85.216.42 as permitted sender)
DomainKey-Signature: a=rsa-sha1; c=nofws;
        d=gmail.com; s=gamma;
        h=mime-version:in-reply-to:references:date:message-id:subject:from:to
         :content-type:content-transfer-encoding;
        b=t/zG1TiHiDRLfApky+NsWKtJLGeV6bBRDriW1HrKHniteBkUOArL4IKM5BgB9YJpKx
         Se9hqfo9p+db9s7WZ3cQ1DZq+UxW4v3VWuEhPtqsBj1QhIX5e2CwWl+b8AIVNaVJ3i+A
         cv7bnnnThmprSAo8NudP5BRzolHHa9eq1amTY=
MIME-Version: 1.0
In-Reply-To: <BANLkTi=yHgLff0CchMDtRAqykBUO=rt0hw@mail.gmail.com>
References: <BANLkTi=ni-2Ug1e7qwPxQ92aok3AHAL17w@mail.gmail.com>
	<BANLkTi=eGDemOkmMMtaUGK=V-1m3qEoruA@mail.gmail.com>
	<BANLkTi=yHgLff0CchMDtRAqykBUO=rt0hw@mail.gmail.com>
Date: Tue, 31 May 2011 22:07:11 -0700
Message-ID: <BANLkTikhkHkwaDEF_Z0+r=hkCi_D+wRvnQ@mail.gmail.com>
Subject: Re: Why do userid & itemid have to be long?
From: Lance Norskog <goksron@gmail.com>
To: user@mahout.apache.org
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: quoted-printable

UserID and ItemID are usually domain-level keys, not generated by the
DB. With some of the movie databases, you get tables of
"user/item/pref/time", "item/moviename/genre", and maybe
"user/geocode".

Lance

On Tue, May 31, 2011 at 9:51 PM, Mike Khristo <mikekhristo@gmail.com> wrote=
:
> Using the 0.6 snapshot + patch 705 (mongodatamodel) from jira (
> https://issues.apache.org/jira/browse/MAHOUT-705), and a test data set wi=
th
> ~300k rows like:
>
> "4cec0a2934ac9fbd2b040000","4d065d5434ac9f5227a12f00",118
>
> It's slowly doing the translations:
> INFO: [+++][MONGO-MAP] Adding Translation =C2=A0 =C2=A0Item ID:
> 4d57d54434ac9fd3570005a2 long_value: 145367
>
> It's doing about 30,000 per hour (and getting slower). That's 8.3/sec.
> 8G ram, 4 virtual cores
>
> With a test data set of 3M preferences, that would take >5 days, just for
> the translation.
>
> Open to ideas/suggestions/"a-ha"-moments. Thanks!
>
>
>
>
> On Tue, May 31, 2011 at 9:15 PM, Ted Dunning <ted.dunning@gmail.com> wrot=
e:
>
>> It makes the internals much cleaner to not repeat this conversion.
>>
>> But how is it that this is taking a long time? =C2=A0String -> lookup sh=
ould not
>> be much longer than an array access, especially if you use the Mahout
>> collections or one of the dictionary types.
>>
>> On Tue, May 31, 2011 at 7:50 PM, Mike Khristo <mikekhristo@gmail.com>
>> wrote:
>>
>> > Rather, how can I use string-based userid/itemid's without having the
>> deal
>> > with the slowness associated with mapping them to a long?
>> >
>> > In the MongoDataModel, for example, significant time/overhead goes int=
o
>> > converting the unique id's to long... =C2=A0I'm still getting my head =
wrapped
>> > around mahout, but this seems like a significant limitation. I have to
>> > assume there's some logic behind the decision to restrict them to long=
,
>> but
>> > i didn't find anything about it in Mahout in Action or the list.
>> >
>> > Thanks.
>> >
>>
>


--=20
Lance Norskog
goksron@gmail.com