Mailing-List: contact dev-help@mahout.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@mahout.apache.org
Date: Sun, 9 Oct 2011 19:31:29 +0000 (UTC)
From: "Sebastian Schelter (Commented) (JIRA)" <jira@apache.org>
To: dev@mahout.apache.org
Message-ID: 
 <498042242.13034.1318188689716.JavaMail.tomcat@hel.zones.apache.org>
In-Reply-To: 
 <1626095318.12706.1318167509713.JavaMail.tomcat@hel.zones.apache.org>
Subject: [jira] [Commented] (MAHOUT-834) rowsimilarityjob doesn't clean it's
 temp dir, and fails when seeing it again
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit


    [ https://issues.apache.org/jira/browse/MAHOUT-834?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13123759#comment-13123759 ] 

Sebastian Schelter commented on MAHOUT-834:
-------------------------------------------

Isn't that the standard behavior of all jobs? Maybe we should a generic option that makes a job delete the temp dir if it already exists.
                
> rowsimilarityjob doesn't clean it's temp dir, and fails when seeing it again
> ----------------------------------------------------------------------------
>
>                 Key: MAHOUT-834
>                 URL: https://issues.apache.org/jira/browse/MAHOUT-834
>             Project: Mahout
>          Issue Type: Bug
>          Components: Integration
>            Reporter: Dan Brickley
>            Priority: Minor
>
> If I do this:
> mahout rowsimilarity --input matrixified/matrix --output sims/ --numberOfColumns 27684 --similarityClassname SIMILARITY_LOGLIKELIHOOD --excludeSelfSimilarity
> then clean my output and rerun,
> rm -rf sims/ # (though this step doesn't even seem needed)
> then try again:
> mahout rowsimilarity --input matrixified/matrix --output sims/ --numberOfColumns 27684 --similarityClassname SIMILARITY_LOGLIKELIHOOD --excludeSelfSimilarity
> The temp files left from the first run make a re-run impossible - we get: "Exception in thread "main" org.apache.hadoop.mapred.FileAlreadyExistsException: Output directory temp/weights already exists".
> Manually deleting the temp directory fixes this.
> I get same behaviour if I explicitly pass in a --tempdir path, e.g.:
> mahout rowsimilarity --input matrixified/matrix --output sims/ --numberOfColumns 27684 --similarityClassname SIMILARITY_LOGLIKELIHOOD --excludeSelfSimilarity --tempDir tmp2/
> Presumably something like HadoopUtil.delete(getConf(),tempDirPath) is needed somewhere?  (and maybe --overwrite too ?)

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira