hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Kai Zheng (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-8095) Allow to configure the system default EC schema
Date Wed, 11 Jan 2017 02:04:58 GMT

    [ https://issues.apache.org/jira/browse/HDFS-8095?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15816849#comment-15816849
] 

Kai Zheng commented on HDFS-8095:
---------------------------------

bq. When you say data corruption, do you mean data loss if the cluster has insufficient #
datanodes or racks?
What I meant is, changing of the system default policy may impact big. But if we can ensure
the policy ID is always associated with the file and persisted in meta, it should be ok even
after the system default policy has changed.

Your understanding reminded me that the user specified policy may be not validated for the
use, as you said, there may be no enough datanodes or racks. So I thought it's a good idea
we should add some necessary check in NN side when user specifying a policy for a file/folder,
or whenever a policy is newly referenced by client requests. We need a separate issue to handle
this guess?

> Allow to configure the system default EC schema
> -----------------------------------------------
>
>                 Key: HDFS-8095
>                 URL: https://issues.apache.org/jira/browse/HDFS-8095
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>            Reporter: Kai Zheng
>            Assignee: Kai Zheng
>              Labels: hdfs-ec-3.0-nice-to-have
>
> As suggested by [~umamaheswararao] and [~vinayrpet] in HDFS-8074, we may desire allowing
to configure the system default EC schema, so in any deployment a cluster admin may be able
to define their own system default one. In the discussion, we have two approaches to configure
the system default schema: 1) predefine it in the {{ecschema-def.xml}} file, making sure it's
not changed; 2) configure the key parameter values as properties in {{core-site.xml}}. Open
this for future consideration in case it's forgotten.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org


Mime
View raw message