hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Eric Yang (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-8811) Support Container Storage Interface (CSI) in YARN
Date Mon, 24 Sep 2018 16:04:00 GMT

    [ https://issues.apache.org/jira/browse/YARN-8811?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16626064#comment-16626064

Eric Yang commented on YARN-8811:

[~cheersyang] {quote}For the comment about object store user API key information, I am not
sure about this point, could you please elaborate.{quote}

In [Hadoop AWS Integration document|https://hadoop.apache.org/docs/current/hadoop-aws/tools/hadoop-aws/index.html],
we need to specify fs.s3a.access.key and fs.s3a.secret.key to connect to aws account.  The
same principal applies to swift fs, and other object store.  Therefore, the CSI specification
should include ability to pass the key information to connect to object store.  In the current
specification, it is also missing source storage information.

Propagation options are required to make sure multiple mount of the same source storage system
can be shared or exclusive mount.  Without this defined, it might be troublesome for source
storage system to decide the locking mechanism.  

> Support Container Storage Interface (CSI) in YARN
> -------------------------------------------------
>                 Key: YARN-8811
>                 URL: https://issues.apache.org/jira/browse/YARN-8811
>             Project: Hadoop YARN
>          Issue Type: New Feature
>            Reporter: Weiwei Yang
>            Assignee: Weiwei Yang
>            Priority: Major
>         Attachments: Support Container Storage Interface(CSI) in YARN_design doc_20180921.pdf
> The Container Storage Interface (CSI) is a vendor neutral interface to bridge Container
Orchestrators and Storage Providers. With the adoption of CSI in YARN, it will be easier to
integrate 3rd party storage systems, and provide the ability to attach persistent volumes
for stateful applications.

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org

View raw message