hadoop-hdfs-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Elek, Marton (JIRA)" <j...@apache.org>
Subject [jira] [Created] (HDDS-1646) Support real persistent in the k8s example files
Date Wed, 05 Jun 2019 12:34:00 GMT
Elek, Marton created HDDS-1646:

             Summary: Support real persistent in the k8s example files 
                 Key: HDDS-1646
                 URL: https://issues.apache.org/jira/browse/HDDS-1646
             Project: Hadoop Distributed Data Store
          Issue Type: Improvement
            Reporter: Elek, Marton
            Assignee: Elek, Marton

Ozone release contains example k8s deployment files to make it easier to deploy Ozone to kubernetes.
As of now we use emptyDir everywhere, we should support the configuration of host volumes
(hostPath or Local Persistent volumes).

The big question here is the default:

* Make the examples easy to start and ephemeral
* Make the examples more safe, by default (but couldn't be started without additional administration).

(Note this conversation is started in the review of HDDS-1508)

Xiaoyu:  Can we support mount hostVolume for datanode daemons?

Marton: Yes, we can.

AFAIK there are two options:
 * using [hostPath](https://kubernetes.io/docs/concepts/storage/volumes/#hostpath)
 * or with [Local PersistentVolumes](https://kubernetes.io/blog/2018/04/13/local-persistent-volumes-beta/)

The first one requires the knowledge of directory names on the host.
The second one is recommended but it requires the creation of PersistentVolumes or install
a PersistentVolume provider

I am not sure what is the best approach, my current proposal is:

 * Use empty dir everywhere to make it easier to start simple ozone cluster
 * Provide simple option to turn on any of theses persistence (the kubernetes files are generated
and the generation can be parametrized)
 * Document how to customize the kubernetes resources files

Summary: it's question of the defaults:
  1. Use a complex, but persistent solution, which may not work out of the box   
  2. Use a simple, but ephemeral solution (as default)

I started to use (2) but I am open to change.

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: hdfs-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-help@hadoop.apache.org

View raw message