From mahout-user-return-1381-apmail-lucene-mahout-user-archive=lucene.apache.org@lucene.apache.org Thu Sep 10 07:27:47 2009 Return-Path: Delivered-To: apmail-lucene-mahout-user-archive@minotaur.apache.org Received: (qmail 63492 invoked from network); 10 Sep 2009 07:27:47 -0000 Received: from hermes.apache.org (HELO mail.apache.org) (140.211.11.3) by minotaur.apache.org with SMTP; 10 Sep 2009 07:27:47 -0000 Received: (qmail 2972 invoked by uid 500); 10 Sep 2009 07:27:46 -0000 Delivered-To: apmail-lucene-mahout-user-archive@lucene.apache.org Received: (qmail 2922 invoked by uid 500); 10 Sep 2009 07:27:46 -0000 Mailing-List: contact mahout-user-help@lucene.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: mahout-user@lucene.apache.org Delivered-To: mailing list mahout-user@lucene.apache.org Received: (qmail 2912 invoked by uid 99); 10 Sep 2009 07:27:46 -0000 Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 10 Sep 2009 07:27:46 +0000 X-ASF-Spam-Status: No, hits=2.2 required=10.0 tests=HTML_MESSAGE,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (nike.apache.org: domain of ted.dunning@gmail.com designates 209.85.210.193 as permitted sender) Received: from [209.85.210.193] (HELO mail-yx0-f193.google.com) (209.85.210.193) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 10 Sep 2009 07:27:36 +0000 Received: by yxe31 with SMTP id 31so8473690yxe.29 for ; Thu, 10 Sep 2009 00:27:16 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:in-reply-to:references :from:date:message-id:subject:to:content-type; bh=QL9daxg/YpX87Ce9kZio7dV3pfBXch7ZSoCOI3dWW1o=; b=OWZuz5JuzECB5JUKA8psZYnA5hrVUERhKfkFnf6uk/p5pIqTmqPJPNxzOwf9r/dzNN hPQUpY9eAs7+/vx4LvXPNj/kG1KmNYCckf/b5pLHNquT6t4y6X8uEkxKbFZYKXv9fgRg UEXtdRPZMvvqj1/kTkjrpw5qQjc8hfbziZYns= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :content-type; b=XSqaEi5WVNYl0GQizW6qS3gcOqRJizy1QvEoaQ77/2JzolQzOsIpgRCsSXoJO/CFyn Ji2vrTWXq2gzRVY71ZskamWIbVwgug2sU+gDBmfpStPM3JYz9zvljiwI/oHSdQR/QgSe 2PJfN29gK4mBCO8QfiFiMJMwyYH0XV3lB+otM= MIME-Version: 1.0 Received: by 10.150.236.18 with SMTP id j18mr2157541ybh.110.1252567636098; Thu, 10 Sep 2009 00:27:16 -0700 (PDT) In-Reply-To: References: <989f35a80909070603h57867000s78f34c83ae6605c5@mail.gmail.com> <989f35a80909091542p3c882630yd97c38a84712f84@mail.gmail.com> From: Ted Dunning Date: Thu, 10 Sep 2009 00:26:56 -0700 Message-ID: Subject: Re: Taste-GenericItemBasedRecommender To: mahout-user@lucene.apache.org Content-Type: multipart/alternative; boundary=000e0cd2d92e01be1d047334210b X-Virus-Checked: Checked by ClamAV on apache.org --000e0cd2d92e01be1d047334210b Content-Type: text/plain; charset=UTF-8 I don't think it is doing more than necessary. And it is surprisingly fast. Matrix like computations have lots of surprises. Even on a single machine, rearranging the loops can have massive effect on speed. And, yes, the product A'A is also sparse, although less sparse than A. Checking for anomalously large values and only keeping those can cause the filtered A'A to be much more sparse than A. On Wed, Sep 9, 2009 at 8:48 PM, Sean Owen wrote: > Now that works... my gut reaction is this is doing way more number > crunching than the existing implementation does. Just producing A'A is > going to do so much work. Yes I realize A is sparse but is the > product? > -- Ted Dunning, CTO DeepDyve --000e0cd2d92e01be1d047334210b--