hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Elek, Marton (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (HADOOP-16091) Create hadoop/ozone docker images with inline build process
Date Mon, 11 Feb 2019 16:48:00 GMT

     [ https://issues.apache.org/jira/browse/HADOOP-16091?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Elek, Marton updated HADOOP-16091:
----------------------------------
    Description: 
This is proposed by [~eyang] in [this|https://lists.apache.org/thread.html/33ac54bdeacb4beb023ebd452464603aaffa095bd104cb43c22f484e@%3Chdfs-dev.hadoop.apache.org%3E]
mailing thread.
{quote}1, 3. There are 38 Apache projects hosting docker images on Docker hub using Apache
Organization. By browsing Apache github mirror. There are only 7 projects using a separate
repository for docker image build. Popular projects official images are not from Apache organization,
such as zookeeper, tomcat, httpd. We may not disrupt what other Apache projects are doing,
but it looks like inline build process is widely employed by majority of projects such as
Nifi, Brooklyn, thrift, karaf, syncope and others. The situation seems a bit chaotic for Apache
as a whole. However, Hadoop community can decide what is best for Hadoop. My preference is
to remove ozone from source tree naming, if Ozone is intended to be subproject of Hadoop for
long period of time. This enables Hadoop community to host docker images for various subproject
without having to check out several source tree to trigger a grand build. However, inline
build process seems more popular than separated process. Hence, I highly recommend making
docker build inline if possible.
{quote}
The main challenges are also discussed in the thread:
{code:java}
3. Technically it would be possible to add the Dockerfile to the source
tree and publish the docker image together with the release by the
release manager but it's also problematic:

{code}
a) there is no easy way to stage the images for the vote
 c) it couldn't be flagged as automated on dockerhub
 d) It couldn't support the critical updates.
 * Updating existing images (for example in case of an ssl bug, rebuild
 all the existing images with exactly the same payload but updated base
 image/os environment)

 * Creating image for older releases (We would like to provide images,
 for hadoop 2.6/2.7/2.7/2.8/2.9. Especially for doing automatic testing
 with different versions).

{code:java}
 {code}
The a) can be solved (as [~eyang] suggested) with using a personal docker image during the
vote and publish it to the dockerhub after the vote (in case the permission can be set by
the INFRA)

Note: based on LEGAL-270 and linked discussion both approaches (inline build process / external
build process) are compatible with the apache release.

Note: HDDS-851 and HADOOP-14898 contains more information about these problems.

  was:
This is proposed by [~eyang] in [this|https://lists.apache.org/thread.html/33ac54bdeacb4beb023ebd452464603aaffa095bd104cb43c22f484e@%3Chdfs-dev.hadoop.apache.org%3E]
mailing thread.

bq. 1, 3. There are 38 Apache projects hosting docker images on Docker hub using Apache Organization.
 By browsing Apache github mirror.  There are only 7 projects using a separate repository
for docker image build.  Popular projects official images are not from Apache organization,
such as zookeeper, tomcat, httpd.  We may not disrupt what other Apache projects are doing,
but it looks like inline build process is widely employed by majority of projects such as
Nifi, Brooklyn, thrift, karaf, syncope and others.  The situation seems a bit chaotic for
Apache as a whole.  However, Hadoop community can decide what is best for Hadoop.  My preference
is to remove ozone from source tree naming, if Ozone is intended to be subproject of Hadoop
for long period of time.  This enables Hadoop community to host docker images for various
subproject without having to check out several source tree to trigger a grand build.  However,
inline build process seems more popular than separated process.  Hence, I highly recommend
making docker build inline if possible.

The main challenges are also discussed in the thread:

{code}
3. Technically it would be possible to add the Dockerfile to the source
tree and publish the docker image together with the release by the
release manager but it's also problematic:

{code}
  a) there is no easy way to stage the images for the vote
  c) it couldn't be flagged as automated on dockerhub
  d) It couldn't support the critical updates.


 * Updating existing images (for example in case of an ssl bug, rebuild
all the existing images with exactly the same payload but updated base
image/os environment)

 * Creating image for older releases (We would like to provide images,
for hadoop 2.6/2.7/2.7/2.8/2.9. Especially for doing automatic testing
with different versions).


{code}

The a) can be solved (as [~eyang] suggested) with using a personal docker image during the
vote and publish it to the dockerhub after the vote (in case the permission can be set by
the INFRA)

Note: based on LEGAL-270 and linked discussion both approaches (inline build process / external
build process) are compatible with the apache release.

Note: HDDS-851 and HADOOP-14898 contains more information about these problems.


> Create hadoop/ozone docker images with inline build process
> -----------------------------------------------------------
>
>                 Key: HADOOP-16091
>                 URL: https://issues.apache.org/jira/browse/HADOOP-16091
>             Project: Hadoop Common
>          Issue Type: New Feature
>            Reporter: Elek, Marton
>            Priority: Major
>
> This is proposed by [~eyang] in [this|https://lists.apache.org/thread.html/33ac54bdeacb4beb023ebd452464603aaffa095bd104cb43c22f484e@%3Chdfs-dev.hadoop.apache.org%3E]
mailing thread.
> {quote}1, 3. There are 38 Apache projects hosting docker images on Docker hub using Apache
Organization. By browsing Apache github mirror. There are only 7 projects using a separate
repository for docker image build. Popular projects official images are not from Apache organization,
such as zookeeper, tomcat, httpd. We may not disrupt what other Apache projects are doing,
but it looks like inline build process is widely employed by majority of projects such as
Nifi, Brooklyn, thrift, karaf, syncope and others. The situation seems a bit chaotic for Apache
as a whole. However, Hadoop community can decide what is best for Hadoop. My preference is
to remove ozone from source tree naming, if Ozone is intended to be subproject of Hadoop for
long period of time. This enables Hadoop community to host docker images for various subproject
without having to check out several source tree to trigger a grand build. However, inline
build process seems more popular than separated process. Hence, I highly recommend making
docker build inline if possible.
> {quote}
> The main challenges are also discussed in the thread:
> {code:java}
> 3. Technically it would be possible to add the Dockerfile to the source
> tree and publish the docker image together with the release by the
> release manager but it's also problematic:
> {code}
> a) there is no easy way to stage the images for the vote
>  c) it couldn't be flagged as automated on dockerhub
>  d) It couldn't support the critical updates.
>  * Updating existing images (for example in case of an ssl bug, rebuild
>  all the existing images with exactly the same payload but updated base
>  image/os environment)
>  * Creating image for older releases (We would like to provide images,
>  for hadoop 2.6/2.7/2.7/2.8/2.9. Especially for doing automatic testing
>  with different versions).
> {code:java}
>  {code}
> The a) can be solved (as [~eyang] suggested) with using a personal docker image during
the vote and publish it to the dockerhub after the vote (in case the permission can be set
by the INFRA)
> Note: based on LEGAL-270 and linked discussion both approaches (inline build process
/ external build process) are compatible with the apache release.
> Note: HDDS-851 and HADOOP-14898 contains more information about these problems.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org


Mime
View raw message