hadoop-mapreduce-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Deepak Vohra <dvohr...@yahoo.com.INVALID>
Subject Re: Where's official Docker image for Hadoop?
Date Tue, 19 Jul 2016 13:33:36 GMT
What is meant by official Hadoop image? Hadoop has several distributions such as HortonWorks,
Cloudera and MapR and they do provide a Docker image.


1. Official image from Cloudera is the quickstart image.
https://hub.docker.com/r/cloudera/quickstart/

2.  From HortonWorks sequenceiq
https://hub.docker.com/r/sequenceiq/hadoop-docker/

3. MapR provides the mapr-sandbox-base
https://hub.docker.com/r/maprtech/mapr-sandbox-base/
--------------------------------------------
On Tue, 7/19/16, Klaus Ma <klaus1982.cn@hotmail.com> wrote:

 Subject: Re: Where's official Docker image for Hadoop?
 To: "Tsuyoshi Ozawa" <ozawa@apache.org>
 Cc: "Deepak Vohra" <dvohra10@yahoo.com>, "Klaus Ma" <klaus1982.cn@gmail.com>,
"user@hadoop.apache.org" <user@hadoop.apache.org>
 Received: Tuesday, July 19, 2016, 5:49 AM
 
 
 Hi Tsuyoshi,
 
 
 
 I have a set of dockerfile at 
 https://github.com/k82cn/outrider/tree/master/kubernetes/imgs/yarn to
 for Apache YARN/HDFS; and I’d like to contribute it to
 upstream if possible.
 
 
 
 Would you also keep me in the
 loop if any discussion?
 
 
 
 Thanks
 Klaus
 
 
 
 
 
 
 
 On
 Jul 19, 2016, at 15:32, Tsuyoshi Ozawa <ozawa@apache.org>
 wrote:
 
 
 
 Hi Klaus,
 
 
 
 Thanks for telling us the request. Currently, the official
 docker
 
 image of Apache Hadoop is not available as Roman mentioned.
 I will
 
 raise this request as discussion.
 
 
 
 Thanks,
 
 - Tsuyoshi
 
 
 
 On Tue, Jul 19, 2016 at 12:29 PM, Deepak Vohra
 
 <dvohra10@yahoo.com.invalid>
 wrote:
 
 A custom
 implementation would have to be developed using some
 container orchestration service such as Kubernetes. Create a
 cluster of Pods (container sets) with different daemons
 running in different Pods and scale the Pods.
  For example, start ResourceManager on one instance and
 NodeManager on multiple instances in the cluster. The
 following blog runs a cluster with NodeManager on multiple
 instances. Kubernetes cluster manager could be added.
 
 
 
 http://www.ibm.com/support/knowledgecenter/STXKQY_4.2.0/com.ibm.spectrum.scale.v4r2.adv.doc/bl1adv_configuringthedockerinstanceandhdfstransparency.htm
 
 
 
 The Docker image used is
 https://hub.docker.com/r/sequenceiq/hadoop-docker/
 
 
 
 But any Docker image could be used.
 
 
 
 
 
 --------------------------------------------
 
 On Mon, 7/18/16, Klaus Ma <klaus1982.cn@gmail.com>
 wrote:
 
 
 
 Subject: Re: Where's official Docker image for
 Hadoop?
 
 To: "Roman Shaposhnik"
 <roman@shaposhnik.org>, "Deepak Vohra"
 <dvohra10@yahoo.com>
 
 Cc: "user@hadoop.apache.org"
 <user@hadoop.apache.org>
 
 Received: Monday, July 18, 2016, 7:00 PM
 
 
 
 #yiv9491280873
 
 #yiv9491280873 -- .yiv9491280873EmailQuote
 
 {margin-left:1pt;padding-left:4pt;border-left:#800000 2px
 
 solid;}#yiv9491280873
 
 
 
 #yiv9491280873 #yiv9491280873 --
 
 p
 
        {margin-top:0;margin-bottom:0;}
 
 #yiv9491280873
 
 
 
 
 
 Thanks for your info.
 
 
 
 
 
 
 
 It seems all daemon are running in one container in
 
 cloudera/quickstart; I'd like to run resourcemanager,
 
 nodemanager in different containter, so I can scale
 
 nodemanager out.
 
 
 
 
 
 
 
 ----
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 Da (Klaus), Ma (马达), PMP®|
 
 Software Architect
 
 
 
 Platform DCOS Development &
 
 Support, STG, IBM GCG
 
 
 
 +86-10-8245 4084 |
 
 madaxa@cn.ibm.com | http://k82.me
 
 
 
 
 
 
 
 
 
 
 
 From: Deepak Vohra
 
 <dvohra10@yahoo.com>
 
 
 
 Sent: Tuesday, July 19, 2016 1:44:34 AM
 
 
 
 To: Roman Shaposhnik
 
 
 
 Cc: user@hadoop.apache.org
 
 
 
 Subject: Re: Where's official Docker image for
 
 Hadoop?
 
 
 
 
 
 
 
 
 
 The   cloudera/quickstart
 
 is the Docker image for Hadoop.
 
 
 
 
 
 
 
 https://hub.docker.com/r/cloudera/quickstart/
 
 
 
 
 
 
 
 Also refer,
 
 
 
 http://www.cloudera.com/documentation/enterprise/5-6-x/topics/quickstart_docker_container.html
 
 
 
 http://blog.cloudera.com/blog/2015/12/docker-is-the-new-quickstart-option-for-apache-hadoop-and-cloudera/
 
 
 
 
 
 
 
 
 
 
 
 --------------------------------------------
 
 
 
 On Mon, 7/18/16, Roman Shaposhnik
 
 <roman@shaposhnik.org> wrote:
 
 
 
 
 
 
 
  Subject: Re: Where's official Docker image for
 
 Hadoop?
 
 
 
  To: "Klaus Ma"
 
 <klaus1982.cn@hotmail.com>
 
 
 
  Cc: "user@hadoop.apache.org"
 
 <user@hadoop.apache.org>
 
 
 
  Received: Monday, July 18, 2016, 5:57 PM
 
 
 
 
 
 
 
  On Mon, Jul 18, 2016 at 5:34 PM,
 
 
 
  Klaus Ma <klaus1982.cn@hotmail.com>
 
 
 
  wrote:
 
 
 
 Hi team,
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 Does anyone
 know where's official docker images?
 
 
 If
 
 
 
  not, I'd like to
 
 
 
 contribute the
 Dockefile for it.
 
 
 
 
 
 
 
 
  I am just curious, what's your use case?
 
 
 
 
 
 
 
  Also, you may want to look at the following "prior
 
 art" in
 
 
 
  the area
 
 
 
  of Hadoop/Docker:
 
 
 
 
 
 http://www.slideshare.net/saintya/trend-micro-big-data-platform-and-apache-bigtop
 
 
 
      https://github.com/trifacta/floating-elephants
 
 
 
 
 
 
 
 BTW, do we
 have official docker hub account for
 
 
 
 
  Hadoop?
 
 
 
 
 
 
 
  It was suggested that it may be useful for ASF to have
 
 an
 
 
 
  official account,
 
 
 
  but nothing materialized AFAIK.
 
 
 
 
 
 
 
  Thanks,
 
 
 
  Roman.
 
 
 
 
 
 
 
  ---------------------------------------------------------------------
 
 
 
  To unsubscribe, e-mail:
 
 user-unsubscribe@hadoop.apache.org
 
 
 
  For additional commands, e-mail:
 
 user-help@hadoop.apache.org
 
 
 
 
 
 
 
 
 
 
 
 
 
 ---------------------------------------------------------------------
 
 To unsubscribe, e-mail:
 user-unsubscribe@hadoop.apache.org
 
 For additional commands, e-mail:
 user-help@hadoop.apache.org
 
 
 
 
 
 
 
 
 
 

---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@hadoop.apache.org
For additional commands, e-mail: user-help@hadoop.apache.org


Mime
View raw message