mahout-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hudson (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (MAHOUT-834) rowsimilarityjob doesn't clean it's temp dir, and fails when seeing it again
Date Mon, 07 May 2012 17:52:50 GMT

    [ https://issues.apache.org/jira/browse/MAHOUT-834?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13269817#comment-13269817
] 

Hudson commented on MAHOUT-834:
-------------------------------

Integrated in Mahout-Quality #1465 (See [https://builds.apache.org/job/Mahout-Quality/1465/])
    MAHOUT-834 rowsimilarityjob doesn't clean it's temp dir, and fails when seeing it again
(Revision 1335032)

     Result = SUCCESS
ssc : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1335032
Files : 
* /mahout/trunk/core/src/main/java/org/apache/mahout/math/hadoop/similarity/cooccurrence/RowSimilarityJob.java

                
> rowsimilarityjob doesn't clean it's temp dir, and fails when seeing it again
> ----------------------------------------------------------------------------
>
>                 Key: MAHOUT-834
>                 URL: https://issues.apache.org/jira/browse/MAHOUT-834
>             Project: Mahout
>          Issue Type: Bug
>          Components: Integration
>    Affects Versions: 0.6, 0.7
>            Reporter: Dan Brickley
>            Assignee: Suneel Marthi
>            Priority: Minor
>             Fix For: 0.7
>
>         Attachments: Mahout-834.patch, Mahout-834.patch
>
>
> If I do this:
> mahout rowsimilarity --input matrixified/matrix --output sims/ --numberOfColumns 27684
--similarityClassname SIMILARITY_LOGLIKELIHOOD --excludeSelfSimilarity
> then clean my output and rerun,
> rm -rf sims/ # (though this step doesn't even seem needed)
> then try again:
> mahout rowsimilarity --input matrixified/matrix --output sims/ --numberOfColumns 27684
--similarityClassname SIMILARITY_LOGLIKELIHOOD --excludeSelfSimilarity
> The temp files left from the first run make a re-run impossible - we get: "Exception
in thread "main" org.apache.hadoop.mapred.FileAlreadyExistsException: Output directory temp/weights
already exists".
> Manually deleting the temp directory fixes this.
> I get same behaviour if I explicitly pass in a --tempdir path, e.g.:
> mahout rowsimilarity --input matrixified/matrix --output sims/ --numberOfColumns 27684
--similarityClassname SIMILARITY_LOGLIKELIHOOD --excludeSelfSimilarity --tempDir tmp2/
> Presumably something like HadoopUtil.delete(getConf(),tempDirPath) is needed somewhere?
 (and maybe --overwrite too ?)

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message