mahout-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Dan Filimon <dangeorge.fili...@gmail.com>
Subject Re: Review Request: MAHOUT-1192 [2]: Speed up Vector Operations
Date Tue, 23 Apr 2013 09:09:16 GMT
Here's [1] a link to the "design document" of the new vector operations so
I can lay out the ideas behind what I'm doing more clearly.

I'd like anyone who can to have a look and comment.

Ted,
I know you'd like me to get back to working on clustering, but currently,
BallKMeans is roughly 2x as slow as Mahout's KMeans.
This is because of centroid.update() for one (it doesn't have "interesting"
properties that are exploited in the current) and also because we're using
we're just doing lots of Vector operations: dot products for projections,
hash codes for unique sets of candidates, equality checks etc.

Here's a snapshot of the hotspots for one run of BallKMeans (20 newsgroups,
unprojected, i.e 20K sparse vectors with 200K dimensions and ~100 nonzero
values):
  Name Time (ms)  org.apache.mahout.math.AbstractVector.dot(Vector) 104069
org.apache.mahout.math.DelegatingVector.hashCode() 30159
org.apache.mahout.math.AbstractVector.assignIterateBoth(Iterator,
Iterator, DoubleDoubleFunction) 13720
org.apache.mahout.math.OrderedIntDoubleMapping.merge(OrderedIntDoubleMapping)
2911  java.lang.ClassLoader.loadClass(String, boolean) 2620
sun.misc.Launcher$AppClassLoader.loadClass(String,
boolean) 2620


You can see that the most time is spent doing Vector operations. Combined,
the first three rows account for 91% of the runtime.

Thanks for your help!

PS: Is this reaching dev@?

[1]
https://docs.google.com/document/d/1g1PjUuvjyh2LBdq2_rKLIcUiDbeOORA1sCJiSsz-JVU/edit#


On Tue, Apr 23, 2013 at 7:53 AM, Robin Anil <robinanil@apache.org> wrote:

> Few more brain dump ideas
>
> For any operation (assign, aggregate) you are trying to do.
> Compute set of "feasible" solutions: Based on all the properties of
> functions
> Choose the solution that minimises the cost.
>
> For this you may need to refactor all the special kinds of iterations and
> aggregations subclassing from AssignOperation and AggregateOperation. Each
> operation would return the cost given two input vectors (based on their
> cost properties)
>
>
>
> ------
> Robin Anil
>
>
>
> On Mon, Apr 22, 2013 at 8:46 PM, Robin Anil <robinanil@apache.org> wrote:
>
>> Strike that. I think the code is very confusing. You will have better
>> luck if you create method for the three implementations. It will make
>> things explicit and easy to debug.
>>
>> public final double getIterationCost() {
>>  return one of {cardinality, 1.2*numNonDefault, numNonDefault}
>> }
>>
>> public final double getLookupCost() {
>>  return one of {1, 1.5(lets say), log_2(numNonDefault)}
>> }
>>
>> All your logic then becomes a minimization of the total cost.
>>
>> This is more extendable than current isDense(), isSequential(),
>> isSparse(), isRandom() and doesn't assume what the underlying
>> implementation is.
>>
>>
>> ------
>> Robin Anil
>>
>>
>>
>> On Mon, Apr 22, 2013 at 8:09 PM, Robin Anil <robinanil@apache.org> wrote:
>>
>>>  Also the calculation for cost is wrong. RASV is HashMap based its cost
>>> O(n) not O(log n)
>>>
>>> ------
>>> Robin Anil
>>>
>>>
>>>
>>> On Mon, Apr 22, 2013 at 4:43 PM, Robin Anil <robinanil@apache.org>wrote:
>>>
>>>>    This is an automatically generated e-mail. To reply, visit:
>>>> https://reviews.apache.org/r/10669/
>>>>    math/src/main/java/org/apache/mahout/math/AbstractVector.java<https://reviews.apache.org/r/10669/diff/2/?file=283185#file283185line56>
(Diff
>>>> revision 2)
>>>>
>>>> None
>>>>
>>>>    55
>>>>
>>>>     boolean commutativeAndAssociative = aggregator.isCommutative() &&
aggregator.isAssociative();
>>>>
>>>>   This should be a method in AbstractFunction interface
>>>>
>>>>
>>>>    math/src/main/java/org/apache/mahout/math/AbstractVector.java<https://reviews.apache.org/r/10669/diff/2/?file=283185#file283185line1037>
(Diff
>>>> revision 2)
>>>>
>>>> {'text': 'public int getNumNonZeroElements() {', 'line': 726, 'expand_offset':
2}
>>>>
>>>> {'text': 'public int getNumNonZeroElements() {', 'line': 927, 'expand_offset':
2}
>>>>
>>>>    942
>>>>
>>>>       // Make all the non-zero values 0.
>>>>
>>>>   vector.clear()
>>>>
>>>>
>>>>    math/src/main/java/org/apache/mahout/math/AbstractVector.java<https://reviews.apache.org/r/10669/diff/2/?file=283185#file283185line1047>
(Diff
>>>> revision 2)
>>>>
>>>> {'text': 'public int getNumNonZeroElements() {', 'line': 726, 'expand_offset':
2}
>>>>
>>>> {'text': 'public int getNumNonZeroElements() {', 'line': 927, 'expand_offset':
2}
>>>>
>>>>    952
>>>>
>>>>       while (it.hasNext()) {
>>>>
>>>>   There is no guarantee of ordered iteration. The updates is unsorted. merge
typically implies that input is sorted
>>>>
>>>>
>>>>    math/src/main/java/org/apache/mahout/math/AbstractVector.java<https://reviews.apache.org/r/10669/diff/2/?file=283185#file283185line1074>
(Diff
>>>> revision 2)
>>>>
>>>> {'text': 'public int getNumNonZeroElements() {', 'line': 726, 'expand_offset':
2}
>>>>
>>>> {'text': 'public int getNumNonZeroElements() {', 'line': 927, 'expand_offset':
2}
>>>>
>>>>    979
>>>>
>>>>         element.set(values[index]);
>>>>
>>>>   What if there are values that are not in the existing data. I dont this
is correct
>>>>
>>>>
>>>>    math/src/main/java/org/apache/mahout/math/AbstractVector.java<https://reviews.apache.org/r/10669/diff/2/?file=283185#file283185line1100>
(Diff
>>>> revision 2)
>>>>
>>>> {'text': 'public int getNumNonZeroElements() {', 'line': 726, 'expand_offset':
2}
>>>>
>>>> {'text': 'public int getNumNonZeroElements() {', 'line': 927, 'expand_offset':
2}
>>>>
>>>>    999
>>>>
>>>>     invalidateCachedLength();
>>>>
>>>>   move invalidateCachedLength to the set method. Instead of dispersing it
everywhere.
>>>>
>>>>
>>>>    math/src/main/java/org/apache/mahout/math/AbstractVector.java<https://reviews.apache.org/r/10669/diff/2/?file=283185#file283185line1115>
(Diff
>>>> revision 2)
>>>>
>>>> None
>>>>
>>>> {'text': '  private Vector assignIterateBoth(Iterator<Element> thisIterator,
Iterator<Element> thatIterator,', 'line': 1070}
>>>>
>>>>     1014
>>>>
>>>>   protected Vector assignSkipZerosIterateOne(Iterator<Element> thisIterator,
Vector other,
>>>>
>>>>   be consistent, this that or thisIterator that Iterator.
>>>>
>>>>
>>>>    math/src/main/java/org/apache/mahout/math/AbstractVector.java<https://reviews.apache.org/r/10669/diff/2/?file=283185#file283185line1171>
(Diff
>>>> revision 2)
>>>>
>>>> None
>>>>
>>>> {'text': '  private Vector assignIterateBoth(Iterator<Element> thisIterator,
Iterator<Element> thatIterator,', 'line': 1070}
>>>>
>>>>     1070
>>>>
>>>>   private Vector assignIterateBoth(Iterator<Element> thisIterator,
Iterator<Element> thatIterator,
>>>>
>>>>   This expects the data to to be sorted. indicate that in a comment. Similarly
comment other places
>>>>
>>>>
>>>>    math/src/main/java/org/apache/mahout/math/function/Functions.java<https://reviews.apache.org/r/10669/diff/2/?file=283202#file283202line900>
(Diff
>>>> revision 2)
>>>>
>>>> None
>>>>
>>>> {'text': '      public boolean isLikeLeftMult() {', 'line': 892}
>>>>
>>>>    896
>>>>
>>>>       /**
>>>>
>>>>   Move this comments to the base class
>>>>
>>>>
>>>> - Robin
>>>>
>>>> On April 22nd, 2013, 8:45 p.m., Dan Filimon wrote:
>>>>   Review request for mahout, Ted Dunning, Sebastian Schelter, and
>>>> Robin Anil.
>>>> By Dan Filimon.
>>>>
>>>> *Updated April 22, 2013, 8:45 p.m.*
>>>> Description
>>>>
>>>> This patch contains code cleaning up AbstractVector and making the operations
as fast as possible while still having a high level interface.
>>>>
>>>> The main changes are in AbstractVector as well as new methods in DoubleDoubleFunction.
>>>>
>>>>   Testing
>>>>
>>>> The vectors test pass but it's likely that the patch in it's current state
is broken as other, unrelated tests (BallKMeans...) are failing.
>>>> Also, my Hadoop conf is broken so I didn't run all the core tests. Anyone?
>>>>
>>>> I can't seem to find the bug, so _please_ have a closer look. It's still
a work in progress.
>>>>
>>>> The benchmarks seem comparable (although there are some jarring diferences
– Minkowski distance seems a lot slower in new-dan-1 than old-trunk-2). It may be however
that this is just variance due to the load of the machine at the time. I'm having trouble
interpreting the benchmarks in general, so anyone who could give me a hand is more than welcome.
>>>>
>>>>   Diffs
>>>>
>>>>    - math/src/main/java/org/apache/mahout/math/AbstractMatrix.java
>>>>    (e12aa38)
>>>>    - math/src/main/java/org/apache/mahout/math/AbstractVector.java
>>>>    (090aa7a)
>>>>    - math/src/main/java/org/apache/mahout/math/Centroid.java (0c42196)
>>>>    - math/src/main/java/org/apache/mahout/math/ConstantVector.java
>>>>    (51d67d4)
>>>>    - math/src/main/java/org/apache/mahout/math/DelegatingVector.java
>>>>    (12220d4)
>>>>    - math/src/main/java/org/apache/mahout/math/DenseVector.java
>>>>    (41c356b)
>>>>    - math/src/main/java/org/apache/mahout/math/FileBasedSparseBinaryMatrix.java
>>>>    (094003b)
>>>>    - math/src/main/java/org/apache/mahout/math/MatrixSlice.java
>>>>    (7f79c96)
>>>>    - math/src/main/java/org/apache/mahout/math/MatrixVectorView.java
>>>>    (af70727)
>>>>    - math/src/main/java/org/apache/mahout/math/NamedVector.java
>>>>    (4b7a41d)
>>>>    - math/src/main/java/org/apache/mahout/math/OrderedIntDoubleMapping.java
>>>>    (650d82d)
>>>>    - math/src/main/java/org/apache/mahout/math/PermutedVectorView.java
>>>>    (d1ea93a)
>>>>    - math/src/main/java/org/apache/mahout/math/RandomAccessSparseVector.java
>>>>    (6f85692)
>>>>    - math/src/main/java/org/apache/mahout/math/SequentialAccessSparseVector.java
>>>>    (21982f9)
>>>>    - math/src/main/java/org/apache/mahout/math/Vector.java (2f8b417)
>>>>    - math/src/main/java/org/apache/mahout/math/VectorView.java
>>>>    (add2a60)
>>>>    - math/src/main/java/org/apache/mahout/math/WeightedVector.java
>>>>    (06fbd05)
>>>>    - math/src/main/java/org/apache/mahout/math/function/DoubleDoubleFunction.java
>>>>    (82b350a)
>>>>    - math/src/main/java/org/apache/mahout/math/function/Functions.java
>>>>    (eb2e42f)
>>>>    - math/src/main/java/org/apache/mahout/math/function/PlusMult.java
>>>>    (60587b1)
>>>>    - math/src/main/java/org/apache/mahout/math/function/TimesFunction.java
>>>>    (8ab0bb1)
>>>>    - math/src/main/java/org/apache/mahout/math/jet/math/Constants.java
>>>>    (53535d6)
>>>>    - math/src/test/java/org/apache/mahout/math/AbstractVectorTest.java
>>>>    (2b11199)
>>>>    - math/src/test/java/org/apache/mahout/math/VectorTest.java
>>>>    (d6d554b)
>>>>
>>>> View Diff <https://reviews.apache.org/r/10669/diff/>
>>>>
>>>
>>>
>>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message