spark-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Burak Yavuz <brk...@gmail.com>
Subject Re: Export BLAS module on Spark MLlib
Date Mon, 30 Nov 2015 22:15:55 GMT
Or you could also use reflection like in this Spark Package:
https://github.com/brkyvz/lazy-linalg/blob/master/src/main/scala/com/brkyvz/spark/linalg/BLASUtils.scala

Best,
Burak

On Mon, Nov 30, 2015 at 12:48 PM, DB Tsai <dbtsai@dbtsai.com> wrote:

> The workaround is have your code in the same package, or write some
> utility wrapper in the same package so you can use them in your code.
> Mostly we implement those BLAS for our own need, and we don't have
> general use-case in mind. As a result, if we open them up prematurely,
> it will add our api maintenance cost. Once it's getting mature, and
> people are asking for them, we will gradually make them public.
>
> Thanks.
>
> Sincerely,
>
> DB Tsai
> ----------------------------------------------------------
> Web: https://www.dbtsai.com
> PGP Key ID: 0xAF08DF8D
>
>
> On Sat, Nov 28, 2015 at 5:20 AM, Sasaki Kai <lewuathe@me.com> wrote:
> > Hello
> >
> > I'm developing a Spark package that manipulates Vector and Matrix for
> > machine learning.
> > This package uses mllib.linalg.Vector and mllib.linalg.Matrix in order to
> > achieve compatible interface to mllib itself. But mllib.linalg.BLAS
> module
> > is private inside spark package. We cannot use BLAS from spark package.
> > Due to this, there is no way to manipulate mllib.linalg.{Vector, Matrix}
> > from spark package side.
> >
> > Is there any reason why BLAS module is not set public?
> > If we cannot use BLAS, what is the reasonable option to manipulate Vector
> > and Matrix from spark package?
> >
> > Regards
> > Kai Sasaki(@Lewuathe)
> >
> >
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@spark.apache.org
> For additional commands, e-mail: dev-help@spark.apache.org
>
>

Mime
View raw message