hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Klaus Ma <klaus1982...@hotmail.com>
Subject Re: Where's official Docker image for Hadoop?
Date Tue, 19 Jul 2016 13:40:40 GMT
I means community version; those docker image are provided by vendors.

——
Da (Klaus) Ma (马达), PMP® | Software  Architect
IBM Spectrum, STG, IBM GCG
+86-10-8245 4084 | klaus1982.cn@gmail.com<mailto:klaus1982.cn@gmail.com> | http://k82.me

On Jul 19, 2016, at 21:33, Deepak Vohra <dvohra10@yahoo.com<mailto:dvohra10@yahoo.com>>
wrote:

What is meant by official Hadoop image? Hadoop has several distributions such as HortonWorks,
Cloudera and MapR and they do provide a Docker image.


1. Official image from Cloudera is the quickstart image.
https://hub.docker.com/r/cloudera/quickstart/

2.  From HortonWorks sequenceiq
https://hub.docker.com/r/sequenceiq/hadoop-docker/

3. MapR provides the mapr-sandbox-base
https://hub.docker.com/r/maprtech/mapr-sandbox-base/
--------------------------------------------
On Tue, 7/19/16, Klaus Ma <klaus1982.cn@hotmail.com> wrote:

Subject: Re: Where's official Docker image for Hadoop?
To: "Tsuyoshi Ozawa" <ozawa@apache.org>
Cc: "Deepak Vohra" <dvohra10@yahoo.com>, "Klaus Ma" <klaus1982.cn@gmail.com>,
"user@hadoop.apache.org" <user@hadoop.apache.org>
Received: Tuesday, July 19, 2016, 5:49 AM


Hi Tsuyoshi,



I have a set of dockerfile at
https://github.com/k82cn/outrider/tree/master/kubernetes/imgs/yarn to
for Apache YARN/HDFS; and I’d like to contribute it to
upstream if possible.



Would you also keep me in the
loop if any discussion?



Thanks
Klaus







On
Jul 19, 2016, at 15:32, Tsuyoshi Ozawa <ozawa@apache.org>
wrote:



Hi Klaus,



Thanks for telling us the request. Currently, the official
docker

image of Apache Hadoop is not available as Roman mentioned.
I will

raise this request as discussion.



Thanks,

- Tsuyoshi



On Tue, Jul 19, 2016 at 12:29 PM, Deepak Vohra

<dvohra10@yahoo.com.invalid>
wrote:

A custom
implementation would have to be developed using some
container orchestration service such as Kubernetes. Create a
cluster of Pods (container sets) with different daemons
running in different Pods and scale the Pods.
 For example, start ResourceManager on one instance and
NodeManager on multiple instances in the cluster. The
following blog runs a cluster with NodeManager on multiple
instances. Kubernetes cluster manager could be added.



http://www.ibm.com/support/knowledgecenter/STXKQY_4.2.0/com.ibm.spectrum.scale.v4r2.adv.doc/bl1adv_configuringthedockerinstanceandhdfstransparency.htm



The Docker image used is
https://hub.docker.com/r/sequenceiq/hadoop-docker/



But any Docker image could be used.





--------------------------------------------

On Mon, 7/18/16, Klaus Ma <klaus1982.cn@gmail.com>
wrote:



Subject: Re: Where's official Docker image for
Hadoop?

To: "Roman Shaposhnik"
<roman@shaposhnik.org>, "Deepak Vohra"
<dvohra10@yahoo.com>

Cc: "user@hadoop.apache.org"
<user@hadoop.apache.org>

Received: Monday, July 18, 2016, 7:00 PM



#yiv9491280873

#yiv9491280873 -- .yiv9491280873EmailQuote

{margin-left:1pt;padding-left:4pt;border-left:#800000 2px

solid;}#yiv9491280873



#yiv9491280873 #yiv9491280873 --

p

       {margin-top:0;margin-bottom:0;}

#yiv9491280873





Thanks for your info.







It seems all daemon are running in one container in

cloudera/quickstart; I'd like to run resourcemanager,

nodemanager in different containter, so I can scale

nodemanager out.







----















Da (Klaus), Ma (马达), PMP®|

Software Architect



Platform DCOS Development &

Support, STG, IBM GCG



+86-10-8245 4084 |

madaxa@cn.ibm.com | http://k82.me











From: Deepak Vohra

<dvohra10@yahoo.com>



Sent: Tuesday, July 19, 2016 1:44:34 AM



To: Roman Shaposhnik



Cc: user@hadoop.apache.org



Subject: Re: Where's official Docker image for

Hadoop?









The   cloudera/quickstart

is the Docker image for Hadoop.







https://hub.docker.com/r/cloudera/quickstart/







Also refer,



http://www.cloudera.com/documentation/enterprise/5-6-x/topics/quickstart_docker_container.html



http://blog.cloudera.com/blog/2015/12/docker-is-the-new-quickstart-option-for-apache-hadoop-and-cloudera/











--------------------------------------------



On Mon, 7/18/16, Roman Shaposhnik

<roman@shaposhnik.org> wrote:







 Subject: Re: Where's official Docker image for

Hadoop?



 To: "Klaus Ma"

<klaus1982.cn@hotmail.com>



 Cc: "user@hadoop.apache.org"

<user@hadoop.apache.org>



 Received: Monday, July 18, 2016, 5:57 PM







 On Mon, Jul 18, 2016 at 5:34 PM,



 Klaus Ma <klaus1982.cn@hotmail.com>



 wrote:



Hi team,














Does anyone
know where's official docker images?


If



 not, I'd like to



contribute the
Dockefile for it.








 I am just curious, what's your use case?







 Also, you may want to look at the following "prior

art" in



 the area



 of Hadoop/Docker:





http://www.slideshare.net/saintya/trend-micro-big-data-platform-and-apache-bigtop



     https://github.com/trifacta/floating-elephants







BTW, do we
have official docker hub account for




 Hadoop?







 It was suggested that it may be useful for ASF to have

an



 official account,



 but nothing materialized AFAIK.







 Thanks,



 Roman.







 ---------------------------------------------------------------------



 To unsubscribe, e-mail:

user-unsubscribe@hadoop.apache.org



 For additional commands, e-mail:

user-help@hadoop.apache.org













---------------------------------------------------------------------

To unsubscribe, e-mail:
user-unsubscribe@hadoop.apache.org

For additional commands, e-mail:
user-help@hadoop.apache.org











Mime
View raw message