mahout-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hudson (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (MAHOUT-1529) Finalize abstraction of distributed logical plans from backend operations
Date Tue, 27 May 2014 22:34:01 GMT

    [ https://issues.apache.org/jira/browse/MAHOUT-1529?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14010431#comment-14010431
] 

Hudson commented on MAHOUT-1529:
--------------------------------

SUCCESS: Integrated in Mahout-Quality #2620 (See [https://builds.apache.org/job/Mahout-Quality/2620/])
MAHOUT-1529 closes PR #1 (dlyubimov: rev 8714a0f722663ea5cb16c14c5b8a01e57574cd93)
* spark/src/main/scala/org/apache/mahout/sparkbindings/drm/plan/OpAtAnyKey.scala
* spark/src/main/scala/org/apache/mahout/sparkbindings/drm/SparkBCast.scala
* spark/src/main/scala/org/apache/mahout/sparkbindings/drm/DrmLikeOps.scala
* spark/src/test/scala/org/apache/mahout/sparkbindings/drm/DrmLikeSuite.scala
* math-scala/src/main/scala/org/apache/mahout/math/drm/logical/OpAx.scala
* math-scala/src/main/scala/org/apache/mahout/math/drm/CacheHint.scala
* spark/src/main/scala/org/apache/mahout/sparkbindings/drm/plan/OpAx.scala
* spark/src/main/scala/org/apache/mahout/sparkbindings/drm/plan/CheckpointAction.scala
* spark/src/main/scala/org/apache/mahout/sparkbindings/blas/AtA.scala
* spark/src/main/scala/org/apache/mahout/sparkbindings/drm/CheckpointedDrmSpark.scala
* math-scala/src/main/scala/org/apache/mahout/math/drm/logical/OpRowRange.scala
* spark/src/main/scala/org/apache/mahout/sparkbindings/drm/plan/OpABt.scala
* math-scala/src/main/scala/org/apache/mahout/math/drm/BCast.scala
* math-scala/src/main/scala/org/apache/mahout/math/drm/logical/OpAtB.scala
* spark/src/main/scala/org/apache/mahout/sparkbindings/drm/DrmRddInput.scala
* spark/src/main/scala/org/apache/mahout/sparkbindings/blas/MapBlock.scala
* math-scala/src/main/scala/org/apache/mahout/math/drm/RLikeDrmOps.scala
* spark/src/test/scala/org/apache/mahout/sparkbindings/blas/AewBSuite.scala
* spark/src/main/scala/org/apache/mahout/sparkbindings/drm/plan/OpAtA.scala
* spark/src/main/scala/org/apache/mahout/sparkbindings/io/MahoutKryoRegistrator.scala
* spark/src/main/scala/org/apache/mahout/sparkbindings/blas/AinCoreB.scala
* spark/src/main/scala/org/apache/mahout/sparkbindings/drm/RLikeDrmOps.scala
* spark/src/main/scala/org/apache/mahout/sparkbindings/drm/DrmLike.scala
* math-scala/src/main/scala/org/apache/mahout/math/drm/DistributedContext.scala
* math-scala/src/main/scala/org/apache/mahout/math/drm/logical/OpAtx.scala
* spark/src/main/scala/org/apache/mahout/sparkbindings/SparkDistributedContext.scala
* spark/src/test/scala/org/apache/mahout/sparkbindings/blas/AtSuite.scala
* math-scala/src/main/scala/org/apache/mahout/math/scalabindings/package.scala
* spark/src/test/scala/org/apache/mahout/sparkbindings/drm/DrmLikeOpsSuite.scala
* spark/src/test/scala/org/apache/mahout/sparkbindings/test/MahoutLocalContext.scala
* math-scala/src/main/scala/org/apache/mahout/math/drm/logical/OpABAnyKey.scala
* spark/src/main/scala/org/apache/mahout/sparkbindings/blas/At.scala
* math-scala/src/main/scala/org/apache/mahout/math/drm/logical/AbstractBinaryOp.scala
* math-scala/pom.xml
* math-scala/src/main/scala/org/apache/mahout/math/scalabindings/decompositions/SSVD.scala
* spark/src/main/scala/org/apache/mahout/sparkbindings/SparkEngine.scala
* spark/src/main/scala/org/apache/mahout/sparkbindings/drm/CacheHint.scala
* spark/src/main/scala/org/apache/mahout/sparkbindings/drm/plan/package.scala
* math-scala/src/main/scala/org/apache/mahout/math/drm/logical/OpAB.scala
* spark/src/test/scala/org/apache/mahout/sparkbindings/blas/ABtSuite.scala
* spark-shell/src/main/scala/org/apache/mahout/sparkbindings/shell/MahoutSparkILoop.scala
* math-scala/src/main/scala/org/apache/mahout/math/drm/DrmLike.scala
* spark/src/main/scala/org/apache/mahout/sparkbindings/drm/plan/OpAtB.scala
* spark/src/main/scala/org/apache/mahout/sparkbindings/drm/plan/AbstractBinaryOp.scala
* spark/src/test/scala/org/apache/mahout/sparkbindings/drm/RLikeDrmOpsSuite.scala
* spark/src/main/scala/org/apache/mahout/sparkbindings/drm/plan/OpTimesRightMatrix.scala
* spark/src/main/scala/org/apache/mahout/sparkbindings/decompositions/DQR.scala
* spark/src/main/scala/org/apache/mahout/sparkbindings/drm/plan/OpAewB.scala
* math-scala/src/main/scala/org/apache/mahout/math/drm/CheckpointedOps.scala
* spark/src/main/scala/org/apache/mahout/sparkbindings/drm/plan/OpABAnyKey.scala
* spark-shell/src/main/scala/org/apache/mahout/sparkbindings/shell/Main.scala
* spark/src/main/scala/org/apache/mahout/sparkbindings/drm/CheckpointedDrm.scala
* math-scala/src/main/scala/org/apache/mahout/math/drm/logical/CheckpointAction.scala
* spark-shell/src/test/mahout/simple.mscala
* math-scala/src/main/scala/org/apache/mahout/math/drm/logical/OpABt.scala
* spark/src/main/scala/org/apache/mahout/sparkbindings/drm/plan/OpAewScalar.scala
* math-scala/src/main/scala/org/apache/mahout/math/drm/logical/OpAtAnyKey.scala
* spark/src/main/scala/org/apache/mahout/sparkbindings/blas/Slicing.scala
* math-scala/src/main/scala/org/apache/mahout/math/drm/decompositions/DSPCA.scala
* spark/src/test/scala/org/apache/mahout/sparkbindings/decompositions/MathSuite.scala
* math-scala/src/main/scala/org/apache/mahout/math/drm/logical/OpAt.scala
* spark/src/main/scala/org/apache/mahout/sparkbindings/drm/CheckpointedDrmSparkOps.scala
* math-scala/src/main/scala/org/apache/mahout/math/drm/logical/OpMapBlock.scala
* spark/src/main/scala/org/apache/mahout/sparkbindings/drm/plan/OpAtx.scala
* spark/src/main/scala/org/apache/mahout/sparkbindings/decompositions/DSSVD.scala
* math-scala/src/main/scala/org/apache/mahout/math/drm/decompositions/DQR.scala
* spark/src/main/scala/org/apache/mahout/sparkbindings/blas/package.scala
* math-scala/src/main/scala/org/apache/mahout/math/drm/logical/OpAewScalar.scala
* spark/src/test/scala/org/apache/mahout/sparkbindings/blas/AtASuite.scala
* spark/src/main/scala/org/apache/mahout/sparkbindings/blas/DrmRddOps.scala
* spark/src/main/scala/org/apache/mahout/sparkbindings/blas/AewB.scala
* spark/src/main/scala/org/apache/mahout/sparkbindings/drm/plan/OpAt.scala
* spark/src/main/scala/org/apache/mahout/sparkbindings/drm/CheckpointedOps.scala
* math-scala/src/main/scala/org/apache/mahout/math/drm/logical/AbstractUnaryOp.scala
* spark/src/main/scala/org/apache/mahout/sparkbindings/drm/package.scala
* math-scala/src/main/scala/org/apache/mahout/math/drm/package.scala
* spark/src/main/scala/org/apache/mahout/sparkbindings/blas/ABt.scala
* math-scala/src/main/scala/org/apache/mahout/math/scalabindings/SSVD.scala
* spark/src/main/scala/org/apache/mahout/sparkbindings/drm/plan/OpTimesLeftMatrix.scala
* spark/src/main/scala/org/apache/mahout/sparkbindings/drm/plan/OpMapBlock.scala
* spark/src/main/scala/org/apache/mahout/sparkbindings/decompositions/DSPCA.scala
* spark/src/main/scala/org/apache/mahout/sparkbindings/package.scala
* spark/src/main/scala/org/apache/mahout/sparkbindings/drm/CheckpointedDrmBase.scala
* spark/pom.xml
* math-scala/src/main/scala/org/apache/mahout/math/drm/decompositions/DSSVD.scala
* spark/src/main/scala/org/apache/mahout/sparkbindings/drm/plan/OpAB.scala
* spark/src/main/scala/org/apache/mahout/sparkbindings/drm/plan/OpRowRange.scala
* math-scala/src/main/scala/org/apache/mahout/math/drm/DrmLikeOps.scala
* math-scala/src/main/scala/org/apache/mahout/math/drm/CheckpointedDrm.scala
* spark/src/main/scala/org/apache/mahout/sparkbindings/blas/Ax.scala
* math-scala/src/main/scala/org/apache/mahout/math/drm/logical/OpTimesLeftMatrix.scala
* math-scala/src/main/scala/org/apache/mahout/math/drm/logical/OpAewB.scala
* math-scala/src/main/scala/org/apache/mahout/math/drm/DistributedEngine.scala
* spark/src/main/scala/org/apache/mahout/sparkbindings/blas/AtB.scala
* spark/src/main/scala/org/apache/mahout/sparkbindings/drm/plan/AbstractUnaryOp.scala
* math-scala/src/main/scala/org/apache/mahout/math/drm/logical/OpTimesRightMatrix.scala
* math-scala/src/main/scala/org/apache/mahout/math/drm/logical/OpAtA.scala


> Finalize abstraction of distributed logical plans from backend operations
> -------------------------------------------------------------------------
>
>                 Key: MAHOUT-1529
>                 URL: https://issues.apache.org/jira/browse/MAHOUT-1529
>             Project: Mahout
>          Issue Type: Improvement
>            Reporter: Dmitriy Lyubimov
>            Assignee: Dmitriy Lyubimov
>             Fix For: 1.0
>
>
> We have a few situations when algorithm-facing API has Spark dependencies creeping in.

> In particular, we know of the following cases:
> -(1) checkpoint() accepts Spark constant StorageLevel directly;-
> -(2) certain things in CheckpointedDRM;-
> -(3) drmParallelize etc. routines in the "drm" and "sparkbindings" package.-
> -(5) drmBroadcast returns a Spark-specific Broadcast object-
> (6) Stratosphere/Flink conceptual api changes.
> *Current tracker:* PR #1 https://github.com/apache/mahout/pull/1 - closed, need new PR
for remaining things once ready.
> *Pull requests are welcome*.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

Mime
View raw message