hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Weiwei Yang (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-11918) Ozone: Encapsulate KSM metadata key into protobuf messages for better (de)serialization
Date Thu, 08 Jun 2017 03:00:23 GMT

    [ https://issues.apache.org/jira/browse/HDFS-11918?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16042115#comment-16042115
] 

Weiwei Yang commented on HDFS-11918:
------------------------------------

I agree. Second thought, we could still use strings as KSM DB keys (instead of protobuf messages),
but adding some encapsulation classes for the read/write. This way we could have higher level
API in {{MetadataManagerImpl}} instead of directly manipulating the database. And have a central
place to parse a string to volume/bucket/objectKey instances, where we can have a better UT
to cover different combinations. For now, let me cancel the patch and revisit this if necessary.
Thanks [~anu], [~xyao].

> Ozone: Encapsulate KSM metadata key into protobuf messages for better (de)serialization
> ---------------------------------------------------------------------------------------
>
>                 Key: HDFS-11918
>                 URL: https://issues.apache.org/jira/browse/HDFS-11918
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>          Components: ozone
>            Reporter: Weiwei Yang
>            Assignee: Weiwei Yang
>            Priority: Critical
>         Attachments: HDFS-11918-HDFS-7240.001.patch
>
>
> There are multiple type of keys stored in KSM database
> # Volume Key
> # Bucket Key
> # Object Key
> # User Key
> Currently they are represented as plain string with some conventions, such as
> # /volume
> # /volume/bucket
> # /volume/bucket/key
> # $user
> this approach makes it so difficult to parse volume/bucket/keys from KSM database. Propose
to encapsulate these types of keys into protobuf messages, and take advantage of protobuf
to serialize(deserialize) classes to byte arrays (and vice versa).



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org


Mime
View raw message