hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Yiqun Lin (JIRA)" <j...@apache.org>
Subject [jira] [Comment Edited] (HDFS-12000) Ozone: Container : Add key versioning support
Date Thu, 06 Jul 2017 16:20:00 GMT

    [ https://issues.apache.org/jira/browse/HDFS-12000?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16076819#comment-16076819
] 

Yiqun Lin edited comment on HDFS-12000 at 7/6/17 4:19 PM:
----------------------------------------------------------

Hi [~cheersyang], thanks for uploading the desgin doc for the oznoe key versioning support.
It looks really good. Just some comments from me:

If Ozone support multiple key versions, I think how to manager different key version datas
should be a major problem we should deal with. The multiple key versions mean that we will
store a lot of datas in ozone.  Actually we don't really need to store some file datas every
time users put in ozone when key version is enabled. A better way is that only when we detect
the change between current key data and uploaded file, then we put this file. Otherwise we
can just update the reference info, generate a new key version, but the actual data is current
file. This is a little similar to HDFS snapshot. This way can save some spaces for ozone.

I have a different opition on the way of deleteing key versions when switch enable multiple
version to disabled state. I think it will still be a hard work for KSM to delete all the
version keys/files even if we use a async way. How about using an expire policy for old key
versions as that have mentioned in the doc? This should be a good way for this case.

Finally, I have a question: if we need to complete this work in the first phase work of Ozone?
I am just curious of this. This feature looks realy usefule for the users. But seems this
should need some more work for complte this. There will be lots of places to have a adjustment
for supporting mutilple key versions.


was (Author: linyiqun):
Hi [~cheersyang], thanks for the desgin doc for the oznoe key versioning support. It looks
really good. Just some comments from me:

If Ozone support multiple key versions, I think how to manager different key version datas
should be a major problem we should deal with. The multiple key versions mean that we will
store a lot of datas in ozone.  Actually we don't really need to store some file datas every
time users put in ozone when key version is enabled. A better way is that only when we detect
the change between current key data and uploaded file, then we put this file. Otherwise we
can just update the reference info, generate a new key version, but the actual data is current
file. This is a little similar to HDFS snapshot. This way can save some spaces for ozone.

I have a different opition on the way of deleteing key versions when switch enable multiple
version to disabled state. I think it will still be a hard work for KSM to delete all the
version keys/files even we use a async way. How about using an expire policy for old key versions
as that have mentioned in the doc? This should be a good way for this case.

Finally, I have a question: if we need to complete this work in the first phase work of Ozone?
I am just curious of this. This feature looks realy usefule for the users. But seems this
should need some more work for complte this. There will be lots of places to have a adjustment
for supporting mutilple key versions.

> Ozone: Container : Add key versioning support
> ---------------------------------------------
>
>                 Key: HDFS-12000
>                 URL: https://issues.apache.org/jira/browse/HDFS-12000
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>          Components: ozone
>    Affects Versions: HDFS-7240
>            Reporter: Anu Engineer
>            Assignee: Chen Liang
>
> The rest interface of ozone supports versioning of keys. This support comes from the
containers and how chunks are managed to support this feature. This JIRA tracks that feature.
Will post a detailed design doc so that we can talk about this feature.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org


Mime
View raw message