hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Elek, Marton (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDDS-40) Separating packaging of Ozone/HDDS from the main Hadoop
Date Fri, 11 May 2018 11:59:00 GMT

    [ https://issues.apache.org/jira/browse/HDDS-40?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16471799#comment-16471799

Elek, Marton commented on HDDS-40:

I just uploaded the latest version

Notable changes:

1. hadoop-dist/target/ozone directory is used as a destination (To keep the version name in
the root folder inside the tar I added a small magic to ozone-dist-tar-stitching to create
a symbolic link + dereference)

2. The usage of the the acceptance tests are simplified after the previous fix. It's more
easy to run the tests from cli. The docker-compose file could be used directly from the acceptance-test/src/test/compose

3. I improved the README

4. I added  two wrapper scripts (acceptance-test/dev-support-bin)
   * If you have local robot installation, you can ./hadoop-ozone/acceptance-test/dev-support/bin/robot-all.sh
   * If you have not: use the mvn command from the README 

> Separating packaging of Ozone/HDDS from the main Hadoop
> -------------------------------------------------------
>                 Key: HDDS-40
>                 URL: https://issues.apache.org/jira/browse/HDDS-40
>             Project: Hadoop Distributed Data Store
>          Issue Type: Improvement
>            Reporter: Elek, Marton
>            Assignee: Elek, Marton
>            Priority: Major
>         Attachments: HDDS-40.001.patch, HDDS-40.002.patch, HDDS-40.003.patch
> According to the community vote, Ozone/Hdds release cycle should be independent from
the Hadoop release cycle.
> To make it possible we need a separated ozone package.
> *The current state:*
> We have just one output tar/directory under hadoop-dist (hadoop-3.2.0). It includes all
the hdfs/yarn/mapreduce/hdds binaries and libraries. (Jar files are put in separated directory).
> The hdds components and hdfs compobebts all could be started from the bin. 
> *Proposed version*
> Create a sepearated hadoop-dist/ozone-2.1.0 which contains only the hdfs AND hdds components.
Both the hdfs namenode and hdds datanode/scm/ksm could be started from the ozone-2.1.0 package.

> Hdds packages would be removed from the original hadoop-3.2.0 directory.
> This is a relatively small change. On further JIRAs we need to :
>  * Create a shaded datanode plugin which could be used with any existing hadoop cluster
>  * Use standalone ObjectStore/Ozone server instead of the Namenode+Datanod plugin.
>  * Add test cases for both the ozone-only and the mixed clusters (ozone + hdfs)

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org

View raw message