I don't think it is doing more than necessary.  And it is surprisingly fast.

Matrix like computations have lots of surprises.  Even on a single machine,
rearranging the loops can have massive effect on speed.

And, yes, the product A'A is also sparse, although less sparse than A.
Checking for anomalously large values and only keeping those can cause the
filtered A'A to be much more sparse than A.

On Wed, Sep 9, 2009 at 8:48 PM, Sean Owen <srowen@gmail.com> wrote:

> Now that works... my gut reaction is this is doing way more number
> crunching than the existing implementation does. Just producing A'A is
> going to do so much work. Yes I realize A is sparse but is the
> product?
>



-- 
Ted Dunning, CTO
DeepDyve