spark-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Reynold Xin <r...@databricks.com>
Subject Re: Using spark MLlib without installing Spark
Date Sun, 22 Nov 2015 04:03:46 GMT
You can use MLlib and Spark directly without "installing anything". Just
run Spark in local mode.


On Sat, Nov 21, 2015 at 4:05 PM, Rad Gruchalski <radek@gruchalski.com>
wrote:

> Bowen,
>
> What Andy is doing in the notebook is a slightly different thing. He’s
> using sbt to bring all spark jars (core, mllib, repl, what have you). You
> could use maven for that. He then creates a repl and submits all the spark
> code into it.
> Pretty sure spark unit tests cover similar uses cases. Maybe not mllib per
> se but this kind of submission.
>
> Kind regards,
> Radek Gruchalski
> radek@gruchalski.com <radek@gruchalski.com>
> de.linkedin.com/in/radgruchalski/
>
>
> *Confidentiality:*This communication is intended for the above-named
> person and may be confidential and/or legally privileged.
> If it has come to you in error you must take no action based on it, nor
> must you copy or show it to anyone; please delete/destroy and inform the
> sender immediately.
>
> On Sunday, 22 November 2015 at 01:01, bowen zhang wrote:
>
> Thanks Rad for info. I looked into the repo and see some .snb file using
> spark mllib. Can you give me a more specific place to look for when
> invoking the mllib functions? What if I just want to invoke some of the ML
> functions in my HelloWorld.java?
>
> ------------------------------
> *From:* Rad Gruchalski <radek@gruchalski.com>
> *To:* bowen zhang <bowenzhangusa@yahoo.com>
> *Cc:* "dev@spark.apache.org" <dev@spark.apache.org>
> *Sent:* Saturday, November 21, 2015 3:43 PM
> *Subject:* Re: Using spark MLlib without installing Spark
>
> Bowen,
>
> One project to look at could be spark-notebook:
> https://github.com/andypetrella/spark-notebook
> It uses Spark you in the way you intend to use it.
> Kind regards,
> Radek Gruchalski
> radek@gruchalski.com <radek@gruchalski.com>
> de.linkedin.com/in/radgruchalski/
>
>
> *Confidentiality:*This communication is intended for the above-named
> person and may be confidential and/or legally privileged.
> If it has come to you in error you must take no action based on it, nor
> must you copy or show it to anyone; please delete/destroy and inform the
> sender immediately.
>
>
> On Sunday, 22 November 2015 at 00:38, bowen zhang wrote:
>
> Hi folks,
> I am a big fan of Spark's Mllib package. I have a java web app where I
> want to run some ml jobs inside the web app. My question is: is there a way
> to just import spark-core and spark-mllib jars to invoke my ML jobs without
> installing the entire Spark package? All the tutorials related Spark seems
> to indicate installing Spark is a pre-condition for this.
>
> Thanks,
> Bowen
>
>
>
>
>
>

Mime
View raw message