hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Weiwei Yang (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-11918) Ozone: Encapsulate KSM metadata key into protobuf messages for better (de)serialization
Date Thu, 08 Jun 2017 03:00:23 GMT

    [ https://issues.apache.org/jira/browse/HDFS-11918?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16042115#comment-16042115

Weiwei Yang commented on HDFS-11918:

I agree. Second thought, we could still use strings as KSM DB keys (instead of protobuf messages),
but adding some encapsulation classes for the read/write. This way we could have higher level
API in {{MetadataManagerImpl}} instead of directly manipulating the database. And have a central
place to parse a string to volume/bucket/objectKey instances, where we can have a better UT
to cover different combinations. For now, let me cancel the patch and revisit this if necessary.
Thanks [~anu], [~xyao].

> Ozone: Encapsulate KSM metadata key into protobuf messages for better (de)serialization
> ---------------------------------------------------------------------------------------
>                 Key: HDFS-11918
>                 URL: https://issues.apache.org/jira/browse/HDFS-11918
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>          Components: ozone
>            Reporter: Weiwei Yang
>            Assignee: Weiwei Yang
>            Priority: Critical
>         Attachments: HDFS-11918-HDFS-7240.001.patch
> There are multiple type of keys stored in KSM database
> # Volume Key
> # Bucket Key
> # Object Key
> # User Key
> Currently they are represented as plain string with some conventions, such as
> # /volume
> # /volume/bucket
> # /volume/bucket/key
> # $user
> this approach makes it so difficult to parse volume/bucket/keys from KSM database. Propose
to encapsulate these types of keys into protobuf messages, and take advantage of protobuf
to serialize(deserialize) classes to byte arrays (and vice versa).

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org

View raw message