# mahout-dev mailing list archives

##### Site index · List index
Message view
Top
From Anand Avati <av...@gluster.org>
Subject Re: Problem of dimensions
Date Sun, 20 Jul 2014 03:56:32 GMT
```On Sat, Jul 19, 2014 at 6:50 PM, Pat Ferrel <pat.ferrel@gmail.com> wrote:

>
> On another thread I’ll send you code that shows A + 1 works with blank
> rows in A.
>

I don't see how that worked for you. See this:

test("DRM addToRowCardinality - will fail") {
val inCoreA = sparse(
0 -> 1 :: 1 -> 2 :: Nil,
0 -> 3 :: 1 -> 4 :: Nil,
0 -> 2 :: 1 -> 0.0 :: Nil
)

val inCoreBControl = sparse(
0 -> 2 :: 1 -> 3 :: Nil,
0 -> 4 :: 1 -> 5 :: Nil,
0 -> 3 :: 1 -> 1 :: Nil,
0 -> 1 :: 1 -> 1 :: Nil,
0 -> 1 :: 1 -> 1 :: Nil
)

val drmA = drmParallelize(inCoreA)
val drmB = (drmA + 1.0).checkpoint()

(drmB.collect - inCoreBControl).norm should be < 1e-3

}

test("DRM addToRowCardinality - wont fail") {
val inCoreA = sparse(
0 -> 1 :: 1 -> 2 :: Nil,
0 -> 3 :: 1 -> 4 :: Nil,
0 -> 2 :: 1 -> 0.0 :: Nil
)

val inCoreBWrong = sparse(
0 -> 2 :: 1 -> 3 :: Nil,
0 -> 4 :: 1 -> 5 :: Nil,
0 -> 3 :: 1 -> 1 :: Nil,
0 -> 0 :: 1 -> 0 :: Nil,
0 -> 0 :: 1 -> 0 :: Nil
)

val drmA = drmParallelize(inCoreA)
val drmB = (drmA + 1.0).checkpoint()

(drmB.collect - inCoreBWrong).norm should be < 1e-3
}

And sure enough, inCoreBControl fails, and inCoreBWrong succeeds:

- DRM addToRowCardinality - will fail *** FAILED ***

2.0 was not less than 0.001 (DrmLikeSuiteBase.scala:116)
- DRM addToRowCardinality - wont fail

BTW this implies rbind will not solve the problem, it is firmly in data
> prep. But until I know the rules I won’t know how to do the right thing.
>

Rbind expects both A and B to have their Int row keys filled from 0 to
nrow-1, which is how they should be ideally.

```
Mime
• Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message