Mailing-List: contact dev-help@mahout.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@mahout.apache.org
Received-SPF: pass (athena.apache.org: domain of ted.dunning@gmail.com
 designates 209.85.160.176 as permitted sender)
MIME-Version: 1.0
In-Reply-To: 
 <CAJBgT0mrtWyq_8Y8DLLOUa10coM5S6qj3ToChkd3hoP11XcNwQ@mail.gmail.com>
References: 
 <CAJBgT0mrtWyq_8Y8DLLOUa10coM5S6qj3ToChkd3hoP11XcNwQ@mail.gmail.com>
From: Ted Dunning <ted.dunning@gmail.com>
Date: Mon, 3 Feb 2014 16:07:35 -0800
Message-ID: 
 <CAJwFCa11pbgEM-z5ejjaev4CvX4-FU=C11kx3c1xYU975ToD3w@mail.gmail.com>
Subject: Re: Optimization of AbstractLogisticRegression.regularize?
To: Mahout Dev List <dev@mahout.apache.org>
Content-Type: multipart/alternative; boundary=001a11c2ddae3f2e6f04f1896fa2

--001a11c2ddae3f2e6f04f1896fa2
Content-Type: text/plain; charset=UTF-8

Optimization of this kind is definitely worthwhile, but 10% improvements
probably are a bit small.  Your difficulty in reproducing the results
indicates how ephemeral small improvements can be.  If you find a 2x
difference, however, we should definitely talk.

It isn't too surprising that inverting loops might make a difference since
it makes a big difference in matrix multiplication as well.  I would worry
a bit about iterating over categories first because that changes the heart
of the algorithm.  If setQuick by itself made a big difference, I might
consider doing it, but I would also expect set to get inlined before long
and optimized into a simple setQuick call.

One of the biggest places to make a serious dent would be in the text
processing that is done in a few places.  I have seen gains of 10x over
naive code that is simply blasting through text or CSV.  If you come up
with a clean and easy to use DFA implementation of that, then it would make
a significant difference.

Beyond that, math library speedups are always welcome.  For instance,
having an extension that offers JBLAS matrices and vectors would be cool
for some applications.


On Mon, Feb 3, 2014 at 12:53 PM, Frank Scholten <frank@frankscholten.nl>wrote:

> Hi all,
>
> I profiled the AbstractLogisticRegression.regularize() method with
> JVisualVM and I could reduce the CPU time by 10% by a doing the following:
>
> * Using setQuick() instead of set()
> * Switching the isSealed() and updateSteps == null check
> * Computing lambda * learningRate outside of the loop
> * Iterating over the categories first and over elements second
>
> I like to experiment a bit more on performance tuning and looking at
> HotSpot and such because I am not sure why certain changes made the
> improvements. Does anyone know how to go from here, in terms of which tools
> to use and what kind of approach to take?
>
> Also, is this optimization even worth it or perhaps there are other areas
> more important in Mahout to optimize? I just ran VisualVM again and now it
> isn't at the top of profiler anymore, now it is
> FeatureVectorEncoder.bytesForString()
>
> Cheers,
>
> Frank
>

--001a11c2ddae3f2e6f04f1896fa2--