mahout-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "jiraposter@reviews.apache.org (Commented) (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (MAHOUT-918) Implement SGD based classifiers using MapReduce
Date Mon, 12 Dec 2011 12:06:32 GMT

    [ https://issues.apache.org/jira/browse/MAHOUT-918?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13167462#comment-13167462
] 

jiraposter@reviews.apache.org commented on MAHOUT-918:
------------------------------------------------------



bq.  On 2011-12-08 07:04:49, Ted Dunning wrote:
bq.  > trunk/core/src/main/java/org/apache/mahout/classifier/sgd/mapreduce/PassiveAggressiveMapper.java,
line 36
bq.  > <https://reviews.apache.org/r/3072/diff/2/?file=63195#file63195line36>
bq.  >
bq.  >     Needs a comment about how this works.

Added comments.


bq.  On 2011-12-08 07:04:49, Ted Dunning wrote:
bq.  > trunk/core/src/main/java/org/apache/mahout/classifier/sgd/mapreduce/PassiveAggressiveMapper.java,
lines 67-75
bq.  > <https://reviews.apache.org/r/3072/diff/2/?file=63195#file63195line67>
bq.  >
bq.  >     This really need a comment.  What is the purpose here?

Added comments.


bq.  On 2011-12-08 07:04:49, Ted Dunning wrote:
bq.  > trunk/core/src/main/java/org/apache/mahout/classifier/sgd/mapreduce/PassiveAggressiveMapper.java,
lines 98-111
bq.  > <https://reviews.apache.org/r/3072/diff/2/?file=63195#file63195line98>
bq.  >
bq.  >     What is this intended to do?  Why?

Added comments.


bq.  On 2011-12-08 07:04:49, Ted Dunning wrote:
bq.  > trunk/core/src/main/java/org/apache/mahout/classifier/sgd/mapreduce/PassiveAggressiveReducer.java,
line 30
bq.  > <https://reviews.apache.org/r/3072/diff/2/?file=63196#file63196line30>
bq.  >
bq.  >     Typo.
bq.  >     
bq.  >     Also, this doesn't say how this works or why it is the way it is.

Fixed the typo and added comments.


bq.  On 2011-12-08 07:04:49, Ted Dunning wrote:
bq.  > trunk/core/src/main/java/org/apache/mahout/classifier/sgd/mapreduce/PassiveAggressiveReducer.java,
line 32
bq.  > <https://reviews.apache.org/r/3072/diff/2/?file=63196#file63196line32>
bq.  >
bq.  >     Shouldn't there be a combiner as well?

A combiner isn't needed because each map task submits one value overall.


bq.  On 2011-12-08 07:04:49, Ted Dunning wrote:
bq.  > trunk/core/src/main/java/org/apache/mahout/classifier/sgd/mapreduce/PassiveAggressiveReducer.java,
line 53
bq.  > <https://reviews.apache.org/r/3072/diff/2/?file=63196#file63196line53>
bq.  >
bq.  >     A comment here about what this weight is would be nice.  Also, how can a double
be a key?  That is tantamount to comparing doubles which is bad.

Added comments. it is not the weight of the classifier but the weight of the weighted average.


bq.  On 2011-12-08 07:04:49, Ted Dunning wrote:
bq.  > trunk/core/src/main/java/org/apache/mahout/classifier/sgd/mapreduce/SGDDriver.java,
line 99
bq.  > <https://reviews.apache.org/r/3072/diff/2/?file=63197#file63197line99>
bq.  >
bq.  >     Where does the InterruptedException come from?

It comes from runIteration function.


bq.  On 2011-12-08 07:04:49, Ted Dunning wrote:
bq.  > trunk/core/src/main/java/org/apache/mahout/classifier/sgd/mapreduce/SGDDriver.java,
lines 110-111
bq.  > <https://reviews.apache.org/r/3072/diff/2/?file=63197#file63197line110>
bq.  >
bq.  >     Use brackets

Added brackets.


bq.  On 2011-12-08 07:04:49, Ted Dunning wrote:
bq.  > trunk/core/src/test/java/org/apache/mahout/classifier/sgd/mapreduce/AdaptiveLogisticRegressionMapReduceTest.java,
line 35
bq.  > <https://reviews.apache.org/r/3072/diff/2/?file=63198#file63198line35>
bq.  >
bq.  >     Should not throw Exception

Added IO Exception and Interrupted Exception.


bq.  On 2011-12-08 07:04:49, Ted Dunning wrote:
bq.  > trunk/core/src/main/java/org/apache/mahout/classifier/sgd/mapreduce/PassiveAggressiveMapper.java,
lines 53-56
bq.  > <https://reviews.apache.org/r/3072/diff/2/?file=63195#file63195line53>
bq.  >
bq.  >     This is nearly duplicated code.  The mapper and reducer should share some code
to avoid inconsistent defaults.

Created a base class which shares the same initialization code.


- issei


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/3072/#review3734
-----------------------------------------------------------


On 2011-12-12 11:51:59, issei yoshida wrote:
bq.  
bq.  -----------------------------------------------------------
bq.  This is an automatically generated e-mail. To reply, visit:
bq.  https://reviews.apache.org/r/3072/
bq.  -----------------------------------------------------------
bq.  
bq.  (Updated 2011-12-12 11:51:59)
bq.  
bq.  
bq.  Review request for mahout.
bq.  
bq.  
bq.  Summary
bq.  -------
bq.  
bq.  MAHOUT-918 Parallelized SGD in MapReduce
bq.  
bq.  
bq.  This addresses bug MAHOUT-918.
bq.      https://issues.apache.org/jira/browse/MAHOUT-918
bq.  
bq.  
bq.  Diffs
bq.  -----
bq.  
bq.    trunk/core/src/main/java/org/apache/mahout/classifier/sgd/PassiveAggressive.java 1213193

bq.    trunk/core/src/main/java/org/apache/mahout/classifier/sgd/mapreduce/AdaptiveLogisticRegressionDriver.java
PRE-CREATION 
bq.    trunk/core/src/main/java/org/apache/mahout/classifier/sgd/mapreduce/AdaptiveLogisticRegressionMapper.java
PRE-CREATION 
bq.    trunk/core/src/main/java/org/apache/mahout/classifier/sgd/mapreduce/AdaptiveLogisticRegressionReducer.java
PRE-CREATION 
bq.    trunk/core/src/main/java/org/apache/mahout/classifier/sgd/mapreduce/LogisticRegressionDriver.java
PRE-CREATION 
bq.    trunk/core/src/main/java/org/apache/mahout/classifier/sgd/mapreduce/LogisticRegressionMapper.java
PRE-CREATION 
bq.    trunk/core/src/main/java/org/apache/mahout/classifier/sgd/mapreduce/LogisticRegressionReducer.java
PRE-CREATION 
bq.    trunk/core/src/main/java/org/apache/mahout/classifier/sgd/mapreduce/PassiveAggressiveDriver.java
PRE-CREATION 
bq.    trunk/core/src/main/java/org/apache/mahout/classifier/sgd/mapreduce/PassiveAggressiveMapper.java
PRE-CREATION 
bq.    trunk/core/src/main/java/org/apache/mahout/classifier/sgd/mapreduce/PassiveAggressiveReducer.java
PRE-CREATION 
bq.    trunk/core/src/main/java/org/apache/mahout/classifier/sgd/mapreduce/SGDDriver.java
PRE-CREATION 
bq.    trunk/core/src/main/java/org/apache/mahout/classifier/sgd/mapreduce/SGDMapper.java
PRE-CREATION 
bq.    trunk/core/src/main/java/org/apache/mahout/classifier/sgd/mapreduce/SGDReducer.java
PRE-CREATION 
bq.    trunk/core/src/test/java/org/apache/mahout/classifier/sgd/mapreduce/AdaptiveLogisticRegressionMapReduceTest.java
PRE-CREATION 
bq.    trunk/core/src/test/java/org/apache/mahout/classifier/sgd/mapreduce/LogisticRegressionMapReduceTest.java
PRE-CREATION 
bq.    trunk/core/src/test/java/org/apache/mahout/classifier/sgd/mapreduce/PassiveAggressiveMapReduceTest.java
PRE-CREATION 
bq.    trunk/core/src/test/java/org/apache/mahout/classifier/sgd/mapreduce/SGDMapReduceTest.java
PRE-CREATION 
bq.  
bq.  Diff: https://reviews.apache.org/r/3072/diff
bq.  
bq.  
bq.  Testing
bq.  -------
bq.  
bq.  
bq.  Thanks,
bq.  
bq.  issei
bq.  
bq.


                
> Implement SGD based classifiers using MapReduce
> -----------------------------------------------
>
>                 Key: MAHOUT-918
>                 URL: https://issues.apache.org/jira/browse/MAHOUT-918
>             Project: Mahout
>          Issue Type: New Feature
>          Components: Classification
>    Affects Versions: 0.6
>            Reporter: issei yoshida
>         Attachments: MAHOUT-918.patch, design.pdf
>
>
> Implement SGD based classifiers (Logistic Regression, Adaptive Logistic regression and
Passive-Aggressive) using MapReduce.
> They are implemented using Iterative Parameter Mixtures algorithm which is referred to
in the following papers.
> http://research.google.com/pubs/pub36948.html
> http://aclweb.org/anthology-new/N/N10/N10-1069.pdf
> http://books.nips.cc/papers/files/nips22/NIPS2009_0345.pdf

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message