hadoop-mapreduce-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Deepak Vohra <dvohr...@yahoo.com.INVALID>
Subject Re: Where's official Docker image for Hadoop?
Date Tue, 19 Jul 2016 13:55:52 GMT

Even the Hadoop documentation refers to the HortonWorks Docker image sequenceiq/hadoop-docker.
https://hadoop.apache.org/docs/r2.7.2/hadoop-yarn/hadoop-yarn-site/DockerContainerExecutor.html

Apache Hadoop develops the Hadoop software, not related technologies such as Docker image.
But a Docker image could be developed using a Dockerfile that downloads and installs a Apache
Hadoop distribution.
--------------------------------------------
On Tue, 7/19/16, Klaus Ma <klaus1982.cn@hotmail.com> wrote:

 Subject: Re: Where's official Docker image for Hadoop?
 To: "Deepak Vohra" <dvohra10@yahoo.com>
 Cc: "user@hadoop.apache.org" <user@hadoop.apache.org>
 Received: Tuesday, July 19, 2016, 6:40 AM
 
 
 I means community version; those docker image are provided
 by vendors.
 
 
 
 
 
 
 
 ——
 
 Da (Klaus) Ma (马达), PMP® | Software  Architect
 
 IBM Spectrum, STG, IBM GCG
 
 +86-10-8245 4084 | klaus1982.cn@gmail.com | http://k82.me
 
 
 
 
 
 
 
 On Jul 19, 2016, at 21:33, Deepak
 Vohra <dvohra10@yahoo.com>
 wrote:
 
 
 
 What is meant by official Hadoop
 image? Hadoop has several distributions such as HortonWorks,
 Cloudera and MapR and they do provide a Docker image.
 
 
 
 
 
 1. Official image from Cloudera is the quickstart image.
 
 https://hub.docker.com/r/cloudera/quickstart/
 
 
 
 2.  From HortonWorks sequenceiq
 
 https://hub.docker.com/r/sequenceiq/hadoop-docker/
 
 
 
 3. MapR provides the mapr-sandbox-base
 
 https://hub.docker.com/r/maprtech/mapr-sandbox-base/
 
 --------------------------------------------
 
 On Tue, 7/19/16, Klaus Ma <klaus1982.cn@hotmail.com>
 wrote:
 
 
 
 Subject: Re: Where's official Docker image for
 Hadoop?
 
 To: "Tsuyoshi Ozawa" <ozawa@apache.org>
 
 Cc: "Deepak Vohra" <dvohra10@yahoo.com>,
 "Klaus Ma" <klaus1982.cn@gmail.com>,
 "user@hadoop.apache.org"
 <user@hadoop.apache.org>
 
 Received: Tuesday, July 19, 2016, 5:49 AM
 
 
 
 
 
 Hi Tsuyoshi,
 
 
 
 
 
 
 
 I have a set of dockerfile at 
 
 https://github.com/k82cn/outrider/tree/master/kubernetes/imgs/yarn to
 
 for Apache YARN/HDFS; and I’d like to contribute it to
 
 upstream if possible.
 
 
 
 
 
 
 
 Would you also keep me in the
 
 loop if any discussion?
 
 
 
 
 
 
 
 Thanks
 
 Klaus
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 On
 
 Jul 19, 2016, at 15:32, Tsuyoshi Ozawa
 <ozawa@apache.org>
 
 wrote:
 
 
 
 
 
 
 
 Hi Klaus,
 
 
 
 
 
 
 
 Thanks for telling us the request. Currently, the
 official
 
 docker
 
 
 
 image of Apache Hadoop is not available as Roman
 mentioned.
 
 I will
 
 
 
 raise this request as discussion.
 
 
 
 
 
 
 
 Thanks,
 
 
 
 - Tsuyoshi
 
 
 
 
 
 
 
 On Tue, Jul 19, 2016 at 12:29 PM, Deepak Vohra
 
 
 
 <dvohra10@yahoo.com.invalid>
 
 wrote:
 
 
 
 A custom
 
 implementation would have to be developed using some
 
 container orchestration service such as Kubernetes. Create
 a
 
 cluster of Pods (container sets) with different daemons
 
 running in different Pods and scale the Pods.
 
  For example, start ResourceManager on one instance and
 
 NodeManager on multiple instances in the cluster. The
 
 following blog runs a cluster with NodeManager on
 multiple
 
 instances. Kubernetes cluster manager could be added.
 
 
 
 
 
 
 
 http://www.ibm.com/support/knowledgecenter/STXKQY_4.2.0/com.ibm.spectrum.scale.v4r2.adv.doc/bl1adv_configuringthedockerinstanceandhdfstransparency.htm
 
 
 
 
 
 
 
 The Docker image used is
 
 https://hub.docker.com/r/sequenceiq/hadoop-docker/
 
 
 
 
 
 
 
 But any Docker image could be used.
 
 
 
 
 
 
 
 
 
 
 
 --------------------------------------------
 
 
 
 On Mon, 7/18/16, Klaus Ma <klaus1982.cn@gmail.com>
 
 wrote:
 
 
 
 
 
 
 
 Subject: Re: Where's official Docker image for
 
 Hadoop?
 
 
 
 To: "Roman Shaposhnik"
 
 <roman@shaposhnik.org>, "Deepak Vohra"
 
 <dvohra10@yahoo.com>
 
 
 
 Cc: "user@hadoop.apache.org"
 
 <user@hadoop.apache.org>
 
 
 
 Received: Monday, July 18, 2016, 7:00 PM
 
 
 
 
 
 
 
 #yiv9491280873
 
 
 
 #yiv9491280873 -- .yiv9491280873EmailQuote
 
 
 
 {margin-left:1pt;padding-left:4pt;border-left:#800000 2px
 
 
 
 solid;}#yiv9491280873
 
 
 
 
 
 
 
 #yiv9491280873 #yiv9491280873 --
 
 
 
 p
 
 
 
        {margin-top:0;margin-bottom:0;}
 
 
 
 #yiv9491280873
 
 
 
 
 
 
 
 
 
 
 
 Thanks for your info.
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 It seems all daemon are running in one container in
 
 
 
 cloudera/quickstart; I'd like to run resourcemanager,
 
 
 
 nodemanager in different containter, so I can scale
 
 
 
 nodemanager out.
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 ----
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 Da (Klaus), Ma (马达), PMP®|
 
 
 
 Software Architect
 
 
 
 
 
 
 
 Platform DCOS Development &
 
 
 
 Support, STG, IBM GCG
 
 
 
 
 
 
 
 +86-10-8245 4084 |
 
 
 
 madaxa@cn.ibm.com | http://k82.me
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 From: Deepak Vohra
 
 
 
 <dvohra10@yahoo.com>
 
 
 
 
 
 
 
 Sent: Tuesday, July 19, 2016 1:44:34 AM
 
 
 
 
 
 
 
 To: Roman Shaposhnik
 
 
 
 
 
 
 
 Cc: user@hadoop.apache.org
 
 
 
 
 
 
 
 Subject: Re: Where's official Docker image for
 
 
 
 Hadoop?
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 The   cloudera/quickstart
 
 
 
 is the Docker image for Hadoop.
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 https://hub.docker.com/r/cloudera/quickstart/
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 Also refer,
 
 
 
 
 
 
 
 http://www.cloudera.com/documentation/enterprise/5-6-x/topics/quickstart_docker_container.html
 
 
 
 
 
 
 
 http://blog.cloudera.com/blog/2015/12/docker-is-the-new-quickstart-option-for-apache-hadoop-and-cloudera/
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 --------------------------------------------
 
 
 
 
 
 
 
 On Mon, 7/18/16, Roman Shaposhnik
 
 
 
 <roman@shaposhnik.org> wrote:
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
  Subject: Re: Where's official Docker image for
 
 
 
 Hadoop?
 
 
 
 
 
 
 
  To: "Klaus Ma"
 
 
 
 <klaus1982.cn@hotmail.com>
 
 
 
 
 
 
 
  Cc: "user@hadoop.apache.org"
 
 
 
 <user@hadoop.apache.org>
 
 
 
 
 
 
 
  Received: Monday, July 18, 2016, 5:57 PM
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
  On Mon, Jul 18, 2016 at 5:34 PM,
 
 
 
 
 
 
 
  Klaus Ma <klaus1982.cn@hotmail.com>
 
 
 
 
 
 
 
  wrote:
 
 
 
 
 
 
 
 Hi team,
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 Does anyone
 
 know where's official docker images?
 
 
 
 
 
 If
 
 
 
 
 
 
 
  not, I'd like to
 
 
 
 
 
 
 
 contribute the
 
 Dockefile for it.
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
  I am just curious, what's your use case?
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
  Also, you may want to look at the following "prior
 
 
 
 art" in
 
 
 
 
 
 
 
  the area
 
 
 
 
 
 
 
  of Hadoop/Docker:
 
 
 
 
 
 
 
 
 
 
 
 http://www.slideshare.net/saintya/trend-micro-big-data-platform-and-apache-bigtop
 
 
 
 
 
 
 
      https://github.com/trifacta/floating-elephants
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 BTW, do we
 
 have official docker hub account for
 
 
 
 
 
 
 
 
 
  Hadoop?
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
  It was suggested that it may be useful for ASF to have
 
 
 
 an
 
 
 
 
 
 
 
  official account,
 
 
 
 
 
 
 
  but nothing materialized AFAIK.
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
  Thanks,
 
 
 
 
 
 
 
  Roman.
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
  ---------------------------------------------------------------------
 
 
 
 
 
 
 
  To unsubscribe, e-mail:
 
 
 
 user-unsubscribe@hadoop.apache.org
 
 
 
 
 
 
 
  For additional commands, e-mail:
 
 
 
 user-help@hadoop.apache.org
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 ---------------------------------------------------------------------
 
 
 
 To unsubscribe, e-mail:
 
 user-unsubscribe@hadoop.apache.org
 
 
 
 For additional commands, e-mail:
 
 user-help@hadoop.apache.org
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@hadoop.apache.org
For additional commands, e-mail: user-help@hadoop.apache.org


Mime
View raw message