spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Matei Zaharia <>
Subject Re: Spark as a Library
Date Tue, 16 Sep 2014 17:30:51 GMT
If you want to run the computation on just one machine (using Spark's local mode), it can probably
run in a container. Otherwise you can create a SparkContext there and connect it to a cluster
outside. Note that I haven't tried this though, so the security policies of the container
might be too restrictive. In that case you'd have to run the app outside and expose an RPC
interface between them.


On September 16, 2014 at 8:17:08 AM, Ruebenacker, Oliver A (




  Suppose I want to use Spark from an application that I already submit to run in another
container (e.g. Tomcat). Is this at all possible? Or do I have to split the app into two components,
and submit one to Spark and one to the other container? In that case, what is the preferred
way for the two components to communicate with each other? Thanks!


     Best, Oliver


Oliver Ruebenacker | Solutions Architect



290 Congress St, 7th Floor | Boston, Massachusetts 02210

P: (617) 728-5582 | ext: 275585 |


This email message and any attachments are intended solely for the use of the addressee. If
you are not the intended recipient, you are prohibited from reading, disclosing, reproducing,
distributing, disseminating or otherwise using this transmission. If you have received this
message in error, please promptly notify the sender by reply email and immediately delete
this message from your system. This message and any attachments may contain information that
is confidential, privileged or exempt from disclosure. Delivery of this message to any person
other than the intended recipient is not intended to waive any right or privilege. Message
transmission is not guaranteed to be secure or free of software viruses.
View raw message