hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ewan Higgs (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-12090) Handling writes from HDFS to Provided storages
Date Mon, 18 Jun 2018 22:44:00 GMT

    [ https://issues.apache.org/jira/browse/HDFS-12090?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16516437#comment-16516437
] 

Ewan Higgs commented on HDFS-12090:
-----------------------------------

We had two phone meetings this week to cover use cases and design and whether/how this work
can be integrated into the SPS.

Attending were [~chris.douglas], [~virajith], [~ehiggs], [~Thomas Demoor], [~sanjay.radia],
[~anu] [~G.UMAMAHESH@GMAIL.COM], [~rakeshr], [~daryn]
h1. Topics
h2. Use Cases

It was reiterated that HDFS-12090 is about tiering and not dispatching between storage tiers
based on paths.
h2. SPS

There was discussion regarding the SPS (HDFS-10285) and whether it should use the replication
monitor. This would allow the Mover to be deprecated and eventually be removed. The SSM (HDFS-7343)
would still likely run outside the NN.
h2. Transfer Block vs New Protocol (for multipart upload)

The MultipartUpload system (HDFS-13186) was described. The DNA_BACKUP command (HDFS-13310)
was also described.
h2. SyncService in NN vs external NN

Anu, Daryn, and Sanjay expressed a strong preference that the SyncService runs as an external
service. This is due to:
 * Memory pressure
 * CPU pressure
 * Instability during early hardening phase
 * Debugability
 * Release cadence

> Handling writes from HDFS to Provided storages
> ----------------------------------------------
>
>                 Key: HDFS-12090
>                 URL: https://issues.apache.org/jira/browse/HDFS-12090
>             Project: Hadoop HDFS
>          Issue Type: New Feature
>            Reporter: Virajith Jalaparti
>            Priority: Major
>         Attachments: HDFS-12090-Functional-Specification.001.pdf, HDFS-12090-Functional-Specification.002.pdf,
HDFS-12090-Functional-Specification.003.pdf, HDFS-12090-design.001.pdf, HDFS-12090.0000.patch,
HDFS-12090.0001.patch
>
>
> HDFS-9806 introduces the concept of {{PROVIDED}} storage, which makes data in external
storage systems accessible through HDFS. However, HDFS-9806 is limited to data being read
through HDFS. This JIRA will deal with how data can be written to such {{PROVIDED}} storages
from HDFS.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org


Mime
View raw message