hadoop-mapreduce-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Klaus Ma <klaus1982...@hotmail.com>
Subject Re: Where's official Docker image for Hadoop?
Date Tue, 19 Jul 2016 12:49:53 GMT
Hi Tsuyoshi,

I have a set of dockerfile at https://github.com/k82cn/outrider/tree/master/kubernetes/imgs/yarn
to for Apache YARN/HDFS; and I’d like to contribute it to upstream if possible.

Would you also keep me in the loop if any discussion?

Thanks
Klaus


On Jul 19, 2016, at 15:32, Tsuyoshi Ozawa <ozawa@apache.org<mailto:ozawa@apache.org>>
wrote:

Hi Klaus,

Thanks for telling us the request. Currently, the official docker
image of Apache Hadoop is not available as Roman mentioned. I will
raise this request as discussion.

Thanks,
- Tsuyoshi

On Tue, Jul 19, 2016 at 12:29 PM, Deepak Vohra
<dvohra10@yahoo.com.invalid<mailto:dvohra10@yahoo.com.invalid>> wrote:
A custom implementation would have to be developed using some container orchestration service
such as Kubernetes. Create a cluster of Pods (container sets) with different daemons running
in different Pods and scale the Pods. For example, start ResourceManager on one instance and
NodeManager on multiple instances in the cluster. The following blog runs a cluster with NodeManager
on multiple instances. Kubernetes cluster manager could be added.

http://www.ibm.com/support/knowledgecenter/STXKQY_4.2.0/com.ibm.spectrum.scale.v4r2.adv.doc/bl1adv_configuringthedockerinstanceandhdfstransparency.htm

The Docker image used is https://hub.docker.com/r/sequenceiq/hadoop-docker/

But any Docker image could be used.


--------------------------------------------
On Mon, 7/18/16, Klaus Ma <klaus1982.cn@gmail.com> wrote:

Subject: Re: Where's official Docker image for Hadoop?
To: "Roman Shaposhnik" <roman@shaposhnik.org>, "Deepak Vohra" <dvohra10@yahoo.com>
Cc: "user@hadoop.apache.org" <user@hadoop.apache.org>
Received: Monday, July 18, 2016, 7:00 PM

#yiv9491280873
#yiv9491280873 -- .yiv9491280873EmailQuote
{margin-left:1pt;padding-left:4pt;border-left:#800000 2px
solid;}#yiv9491280873

#yiv9491280873 #yiv9491280873 --
p
       {margin-top:0;margin-bottom:0;}
#yiv9491280873


Thanks for your info.



It seems all daemon are running in one container in
cloudera/quickstart; I'd like to run resourcemanager,
nodemanager in different containter, so I can scale
nodemanager out.



----







Da (Klaus), Ma (马达), PMP®|
Software Architect

Platform DCOS Development &
Support, STG, IBM GCG

+86-10-8245 4084 |
madaxa@cn.ibm.com | http://k82.me





From: Deepak Vohra
<dvohra10@yahoo.com>

Sent: Tuesday, July 19, 2016 1:44:34 AM

To: Roman Shaposhnik

Cc: user@hadoop.apache.org

Subject: Re: Where's official Docker image for
Hadoop?




The   cloudera/quickstart
is the Docker image for Hadoop.



https://hub.docker.com/r/cloudera/quickstart/



Also refer,

http://www.cloudera.com/documentation/enterprise/5-6-x/topics/quickstart_docker_container.html

http://blog.cloudera.com/blog/2015/12/docker-is-the-new-quickstart-option-for-apache-hadoop-and-cloudera/





--------------------------------------------

On Mon, 7/18/16, Roman Shaposhnik
<roman@shaposhnik.org> wrote:



 Subject: Re: Where's official Docker image for
Hadoop?

 To: "Klaus Ma"
<klaus1982.cn@hotmail.com>

 Cc: "user@hadoop.apache.org"
<user@hadoop.apache.org>

 Received: Monday, July 18, 2016, 5:57 PM



 On Mon, Jul 18, 2016 at 5:34 PM,

 Klaus Ma <klaus1982.cn@hotmail.com>

 wrote:

Hi team,





Does anyone know where's official docker images?
If

 not, I'd like to

contribute the Dockefile for it.



 I am just curious, what's your use case?



 Also, you may want to look at the following "prior
art" in

 the area

 of Hadoop/Docker:


http://www.slideshare.net/saintya/trend-micro-big-data-platform-and-apache-bigtop

     https://github.com/trifacta/floating-elephants



BTW, do we have official docker hub account for

 Hadoop?



 It was suggested that it may be useful for ASF to have
an

 official account,

 but nothing materialized AFAIK.



 Thanks,

 Roman.



 ---------------------------------------------------------------------

 To unsubscribe, e-mail:
user-unsubscribe@hadoop.apache.org

 For additional commands, e-mail:
user-help@hadoop.apache.org






---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@hadoop.apache.org
For additional commands, e-mail: user-help@hadoop.apache.org


Mime
View raw message