Mailing-List: contact issues-help@spark.apache.org; run by ezmlm
Precedence: bulk
Date: Fri, 20 May 2016 02:45:13 +0000 (UTC)
From: "Nicholas Chammas (JIRA)" <jira@apache.org>
To: issues@spark.apache.org
Message-ID: <JIRA.12746250.1412627993000.243950.1463712313324@Atlassian.JIRA>
In-Reply-To: <JIRA.12746250.1412627993000@Atlassian.JIRA>
References: <JIRA.12746250.1412627993000@Atlassian.JIRA> <JIRA.12746250.1412627993916@arcas>
Subject: [jira] [Commented] (SPARK-3821) Develop an automated way of
 creating Spark images (AMI, Docker, and others)
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit
archived-at: Fri, 20 May 2016 02:45:15 -0000


    [ https://issues.apache.org/jira/browse/SPARK-3821?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15292599#comment-15292599 ] 

Nicholas Chammas commented on SPARK-3821:
-----------------------------------------

You can deploy Spark today on Docker just fine. It's just that Spark itself does not maintain any official Dockerfiles and likely never will since the project is actually trying to push deployment stuff outside the main project (hence why spark-ec2 was moved out; you will not see spark-ec2 in the official docs once Spark 2.0 comes out). You may be more interested in the Apache Big Top project, which focuses on big data system deployment (including Spark) and may have stuff for Docker specifically. 

Mesos is a separate matter, because it's a resource manager (analogous to YARN) that integrates with Spark at a low level.

If you still think Spark should host and maintain an official Dockerfile and Docker images that are suitable for production use, please open a separate issue. I think the maintainers will reject it on the grounds that I have explained here, though. (Can't say for sure; after all I'm just a random contributor.)

> Develop an automated way of creating Spark images (AMI, Docker, and others)
> ---------------------------------------------------------------------------
>
>                 Key: SPARK-3821
>                 URL: https://issues.apache.org/jira/browse/SPARK-3821
>             Project: Spark
>          Issue Type: Improvement
>          Components: Build, EC2
>            Reporter: Nicholas Chammas
>            Assignee: Nicholas Chammas
>         Attachments: packer-proposal.html
>
>
> Right now the creation of Spark AMIs or Docker containers is done manually. With tools like [Packer|http://www.packer.io/], we should be able to automate this work, and do so in such a way that multiple types of machine images can be created from a single template.


--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org