hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Allen Wittenauer (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HADOOP-13397) Add dockerfile for Hadoop
Date Wed, 27 Jul 2016 17:27:20 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-13397?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15396023#comment-15396023

Allen Wittenauer commented on HADOOP-13397:

I had a long discussion yesterday with some folks about this topic, especially around how
to make it something consumable for a wide variety of installation-types.  One of the big
asks was to make it work as a self-contained Dockerfile (so no COPY/ADD, RUNs can only reference
things already inside the image, etc, etc.) as much as possible to allow the Dockerfile to
be used by some other service and/or the basis of adding more content.  This means if I'm
using a configuration service such as bcfg2 or puppet, it would be able to put down the necessary
components at docker build or docker run time.  If I'm using something that takes a supplied
tar ball, then COPY is unavoidable. [1]  It also means it's not a "do everything imaginable"
feature like HBASE-12721 .  One would still need something to actually launch the containers
and give some control information.

I've been playing around a bit and have a simple prototype built based upon those discussions
and what I've seen in Klaus' github repo. I'm going to clean it up and flesh it out a bit
and probably post a patch in week or so, time permitting. (i.e., remove all the hard codes
and start making it take options... haha.)

[1] I mean, we *could* do something like base64 encode the tar ball and extract, but that
seems a little extreme. ;)

> Add dockerfile for Hadoop
> -------------------------
>                 Key: HADOOP-13397
>                 URL: https://issues.apache.org/jira/browse/HADOOP-13397
>             Project: Hadoop Common
>          Issue Type: Bug
>            Reporter: Klaus Ma
>            Assignee: Allen Wittenauer
> For now, there's no community version Dockerfile in Hadoop; most of docker images are
provided by vendor, e.g. 
> 1. Cloudera's image: https://hub.docker.com/r/cloudera/quickstart/
> 2.  From HortonWorks sequenceiq: https://hub.docker.com/r/sequenceiq/hadoop-docker/
> 3. MapR provides the mapr-sandbox-base: https://hub.docker.com/r/maprtech/mapr-sandbox-base/
> The proposal of this JIRA is to provide a community version Dockerfile in Hadoop, and
here's some requirement:
> 1. Seperated docker image for master & agents, e.g. resource manager & node manager
> 2. Default configuration to start master & agent instead of configurating manually
> 3. Start Hadoop process as no-daemon
> Here's my dockerfile to start master/agent: https://github.com/k82cn/outrider/tree/master/kubernetes/imgs/yarn
> I'd like to contribute it after polishing :).
> Email Thread : http://mail-archives.apache.org/mod_mbox/hadoop-user/201607.mbox/%3CSG2PR04MB162977CFE150444FA022510FB6370%40SG2PR04MB1629.apcprd04.prod.outlook.com%3E

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org

View raw message