Mailing-List: contact dev-help@mahout.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@mahout.apache.org
Received-SPF: pass (nike.apache.org: domain of ted.dunning@gmail.com
 designates 209.85.223.176 as permitted sender)
MIME-Version: 1.0
In-Reply-To: <51E5646F.3010607@uowmail.edu.au>
References: 
 <CAFj5n5E3Z4n8CbZqzugzD-Ljn-Kib2L42mnzCQMTerRgjk8uJQ@mail.gmail.com>
 <51E4D4A1.3040505@googlemail.com> <51E5646F.3010607@uowmail.edu.au>
From: Ted Dunning <ted.dunning@gmail.com>
Date: Tue, 16 Jul 2013 11:45:52 -0700
Message-ID: 
 <CAJwFCa264A7HRO4YGsT5ptQCfxQC8HBcR_0NeXomA_PF__sEHg@mail.gmail.com>
Subject: Re: Regarding Online Recommenders
To: Mahout Dev List <dev@mahout.apache.org>
Content-Type: multipart/alternative; boundary=001a113322acc0e61004e1a56480

--001a113322acc0e61004e1a56480
Content-Type: text/plain; charset=UTF-8

Netflix is a small dataset.  12G for that seems quite excessive.

Note also that this is before you have done any work.

Ideally, 100million observations should take << 1GB.

On Tue, Jul 16, 2013 at 8:19 AM, Peng Cheng <pc175@uowmail.edu.au> wrote:

> The second idea is indeed splendid, we should separate time-complexity
> first and space-complexity first implementation. What I'm not quite sure,
> is that if we really need to create two interfaces instead of one.
> Personally, I think 12G heap space is not that high right? Most new laptop
> can already handle that (emphasis on laptop). And if we replace hash map
> (the culprit of high memory consumption) with list/linkedList, it would
> simply degrade time complexity for a linear search to O(n), not too bad
> either. The current DataModel is a result of careful thoughts and has
> underwent extensive test, it is easier to expand on top of it instead of
> subverting it.

--001a113322acc0e61004e1a56480--