hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Weiwei Yang (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-11781) Ozone: KSM: Add deleteKey
Date Thu, 01 Jun 2017 16:02:04 GMT

    [ https://issues.apache.org/jira/browse/HDFS-11781?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16033202#comment-16033202
] 

Weiwei Yang commented on HDFS-11781:
------------------------------------

I am a bit hesitated on when is the right time to remove the key from KSM database, that is
to say the time a key becomes to be invisible for a client. We might want to take a look at
HDFS-11886, similar problem. Delete a key generally took 2 steps 

# Delete a key from KSM namespace
# Delete the key data (object value) from datanodes via SCM

the second step may fail on some sanity checks before actual jobs, such as

# Node is in chill mode
# FAILED_TO_FIND_BLOCK
# ...

in such cases, we cannot reach the point that SCM sends async delete command to datanodes,
which means it becomes to be an orphan block and will not be removed forever. Unless we have
a background thread to scan orphan blocks and cleanup them periodically. Or since SCM layer
state is not changed, we can try-catch such exceptions and make the deleteKey fail, do not
commit that to KSM (or a simply rollback). We may want another JIRA to track this issue, similar
to HDFS-11886. [~yuanbo], [~xyao], [~anu], what's your position?

> Ozone: KSM: Add deleteKey
> -------------------------
>
>                 Key: HDFS-11781
>                 URL: https://issues.apache.org/jira/browse/HDFS-11781
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>          Components: ozone
>    Affects Versions: HDFS-7240
>            Reporter: Anu Engineer
>            Assignee: Yuanbo Liu
>         Attachments: HDFS-11781-HDFS-7240.001.patch, HDFS-11781-HDFS-7240.002.patch
>
>
> Add support for removing a key.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org


Mime
View raw message