Let me try to wrap my head around it........
On Fri, Apr 5, 2013 at 8:07 AM, Sean Owen <srowen@gmail.com> wrote:
> OK yes you're on to something here. I should clarify. Koobas you are
> right that the ALS algorithm itself is fine here as far as my
> knowledge takes me. The thing it inverts to solve for a row of X is
> something like (Y' * Cu * Y + lambda * I). No problem there, and
> indeed I see why the regularization term is part of that.
>
> I'm talking about a later step, after the factorization. You get a new
> row in A and want to solve A = X*Y' for X, given the current Y. (And
> vice versa). I'm using a QR decomposition for that, but not to
> directly solve the system (and this may be the issue), but instead to
> compute and save off (Y' * Y)^1 so that we can figure A * Y *
> (Y'*Y)^1 very fast at runtime. That is to say the problem centers
> around the inverse of Y'*Y and in this example, it does not even
> exist.
>
> I am not sure it's just a numerical precision thing since using an SVD
> to get the inverse gives the same result.
>
> But I certainly have examples where the data (A) is most certainly
> rank >> k and get this bad behavior  for example, when lambda is
> very *high*.
>
>
> On Fri, Apr 5, 2013 at 6:57 AM, Ted Dunning <ted.dunning@gmail.com> wrote:
> > On Fri, Apr 5, 2013 at 2:40 AM, Koobas <koobas@gmail.com> wrote:
> >
> >> Anyways, I saw no particular reason for the method to fail with k
> >> approaching or exceeding m and n.
> >> It does if there is no regularization.
> >> But with regularization in place, k can be pretty much anything.
> >>
> >
> > Ahh... this is an important point and it should handle all of the issues
> of
> > poor conditioning.
> >
> > The regularizer takes the rank deficient A and makes it reasonably well
> > conditioned. How well conditioned depends on the choice of lambda, the
> > regularizing scale constant.
>
