hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (JIRA)" <j...@apache.org>
Subject [jira] [Work logged] (HDDS-1646) Support real persistence in the k8s example files
Date Tue, 11 Jun 2019 12:44:00 GMT

     [ https://issues.apache.org/jira/browse/HDDS-1646?focusedWorklogId=257683&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-257683
]

ASF GitHub Bot logged work on HDDS-1646:
----------------------------------------

                Author: ASF GitHub Bot
            Created on: 11/Jun/19 12:43
            Start Date: 11/Jun/19 12:43
    Worklog Time Spent: 10m 
      Work Description: elek commented on pull request #945: HDDS-1646. Support real persistence
in the k8s example files 
URL: https://github.com/apache/hadoop/pull/945
 
 
   Ozone release contains example k8s deployment files to make it easier to deploy Ozone to
kubernetes. As of now we use emptyDir everywhere, we should support the configuration of host
volumes (hostPath or Local Persistent volumes).
   
   The big question here is the default:
   
   * Make the examples easy to start and ephemeral
   * Make the examples more safe, by default (but couldn't be started without additional administration).
   
   (Note this conversation is started in the review of HDDS-1508)
   
   Xiaoyu:  Can we support mount hostVolume for datanode daemons?
   
   Marton: Yes, we can.
   
   AFAIK there are two options:
    * using [hostPath](https://kubernetes.io/docs/concepts/storage/volumes/#hostpath)
    * or with [Local PersistentVolumes](https://kubernetes.io/blog/2018/04/13/local-persistent-volumes-beta/)
   
   The first one requires the knowledge of directory names on the host.
   The second one is recommended but it requires the creation of PersistentVolumes or install
a PersistentVolume provider
   
   I am not sure what is the best approach, my current proposal is:
   
    * Use empty dir everywhere to make it easier to start simple ozone cluster
    * Provide simple option to turn on any of theses persistence (the kubernetes files are
generated and the generation can be parametrized)
    * Document how to customize the kubernetes resources files
   
   Summary: it's question of the defaults:
    
     1. Use a complex, but persistent solution, which may not work out of the box   
     2. Use a simple, but ephemeral solution (as default)
   
   I started to use (2) but I am open to change.
   
   
   
   See: https://issues.apache.org/jira/browse/HDDS-1646
 
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Issue Time Tracking
-------------------

            Worklog Id:     (was: 257683)
            Time Spent: 10m
    Remaining Estimate: 0h

> Support real persistence in the k8s example files 
> --------------------------------------------------
>
>                 Key: HDDS-1646
>                 URL: https://issues.apache.org/jira/browse/HDDS-1646
>             Project: Hadoop Distributed Data Store
>          Issue Type: Improvement
>            Reporter: Elek, Marton
>            Assignee: Elek, Marton
>            Priority: Critical
>              Labels: pull-request-available
>          Time Spent: 10m
>  Remaining Estimate: 0h
>
> Ozone release contains example k8s deployment files to make it easier to deploy Ozone
to kubernetes. As of now we use emptyDir everywhere, we should support the configuration of
host volumes (hostPath or Local Persistent volumes).
> The big question here is the default:
> * Make the examples easy to start and ephemeral
> * Make the examples more safe, by default (but couldn't be started without additional
administration).
> (Note this conversation is started in the review of HDDS-1508)
> Xiaoyu:  Can we support mount hostVolume for datanode daemons?
> Marton: Yes, we can.
> AFAIK there are two options:
>  * using [hostPath](https://kubernetes.io/docs/concepts/storage/volumes/#hostpath)
>  * or with [Local PersistentVolumes](https://kubernetes.io/blog/2018/04/13/local-persistent-volumes-beta/)
> The first one requires the knowledge of directory names on the host.
> The second one is recommended but it requires the creation of PersistentVolumes or install
a PersistentVolume provider
> I am not sure what is the best approach, my current proposal is:
>  * Use empty dir everywhere to make it easier to start simple ozone cluster
>  * Provide simple option to turn on any of theses persistence (the kubernetes files are
generated and the generation can be parametrized)
>  * Document how to customize the kubernetes resources files
> Summary: it's question of the defaults:
>  
>   1. Use a complex, but persistent solution, which may not work out of the box   
>   2. Use a simple, but ephemeral solution (as default)
> I started to use (2) but I am open to change.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org


Mime
View raw message