hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Kai Zheng (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-7343) A comprehensive and flexible storage policy engine
Date Tue, 04 Nov 2014 06:21:33 GMT

    [ https://issues.apache.org/jira/browse/HDFS-7343?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14195766#comment-14195766

Kai Zheng commented on HDFS-7343:

For [~apurtell]'s [comments|https://issues.apache.org/jira/browse/HDFS-7285?focusedCommentId=14192703&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14192703]:
bq. If a user (application) is aware - through its own statistics - of data temperature, it
should be possible to hint this to the policy engine.
It makes sense. Maybe we could make use of the suggested "force" xattr for this?

bq. How do applications plug into the policy engine machinery?
IMO, user customized policy engine maybe conflict and fight with existing policies. We could
make the policy engine flexible and configurable enough for the requirement. 

bq. Why wouldn't the policy engine have the last word? I.e. (file attributes and metadata,
data temperature, cluster configuration metadata) -> (policy engine)
I quite agree. The policy would have the last word, assembling and considering all kinds of
aspects, preferences and conditions, and make the choice. 

> A comprehensive and flexible storage policy engine
> --------------------------------------------------
>                 Key: HDFS-7343
>                 URL: https://issues.apache.org/jira/browse/HDFS-7343
>             Project: Hadoop HDFS
>          Issue Type: Improvement
>          Components: namenode
>            Reporter: Kai Zheng
> As discussed in HDFS-7285, it would be better to have a comprehensive and flexible storage
policy engine considering file attributes, metadata, data temperature, storage type, EC codec,
available hardware capabilities, user/application preference and etc.

This message was sent by Atlassian JIRA

View raw message