mahout-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Pat Ferrel (JIRA)" <j...@apache.org>
Subject [jira] [Comment Edited] (MAHOUT-1464) Cooccurrence Analysis on Spark
Date Mon, 14 Apr 2014 21:28:26 GMT

    [ https://issues.apache.org/jira/browse/MAHOUT-1464?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13968864#comment-13968864
] 

Pat Ferrel edited comment on MAHOUT-1464 at 4/14/14 9:28 PM:
-------------------------------------------------------------

I think IDEA forces some things to run local so it can keep track of threads or something.
Seems to work correctly with Spark but not HDFS. There are ways to remote debug with it so
it separates processes but I don't need you to help me with IDEA.

Seems easier to answer: How do I run this from the CLI? Let's get IDEA out of the picture.
I bet it will just work.

We need a way to run these from the CLI via cron or scripts anyway, right?

Using spark-class I get no errors but no output either. It doesn't create the same Application
name so I must be using it wrong. Will look later today.


was (Author: pferrel):
I think IDEA forces some things to run local so it can keep track of threads or something.
Seems to work correctly with Spark but not HDFS. There are ways to remote debug with it so
it separates processes but I don't need you to help me with IDEA.

Seems easier to answer: How do I run this from the CLI? Let's get IDEA out of the picture.
I be it will just work.

We need a way to run these from the CLI via cron or scripts anyway, right?

Using spark-class I get no errors but no output either. It doesn't create the same Application
name so I must be using it wrong. Will look later today.

> Cooccurrence Analysis on Spark
> ------------------------------
>
>                 Key: MAHOUT-1464
>                 URL: https://issues.apache.org/jira/browse/MAHOUT-1464
>             Project: Mahout
>          Issue Type: Improvement
>          Components: Collaborative Filtering
>         Environment: hadoop, spark
>            Reporter: Pat Ferrel
>            Assignee: Sebastian Schelter
>             Fix For: 1.0
>
>         Attachments: MAHOUT-1464.patch, MAHOUT-1464.patch, MAHOUT-1464.patch, MAHOUT-1464.patch,
MAHOUT-1464.patch, MAHOUT-1464.patch, run-spark-xrsj.sh
>
>
> Create a version of Cooccurrence Analysis (RowSimilarityJob with LLR) that runs on Spark.
This should be compatible with Mahout Spark DRM DSL so a DRM can be used as input. 
> Ideally this would extend to cover MAHOUT-1422. This cross-cooccurrence has several applications
including cross-action recommendations. 



--
This message was sent by Atlassian JIRA
(v6.2#6252)

Mime
View raw message