hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Chen Liang (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-11154) Block Storage : store server state to persistent storage
Date Mon, 21 Nov 2016 22:29:59 GMT

    [ https://issues.apache.org/jira/browse/HDFS-11154?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15684943#comment-15684943
] 

Chen Liang commented on HDFS-11154:
-----------------------------------

Thanks [~anu] for coming up with this scenario!

I haven't thought about this scenario. But I think even in this race condition it is fine
: if a create and a delete volume call are already racing anyway, there is no way to determine
what should be the right outcome. The only issue here is that they both return successfully
but the volume may or may not be in persistent store. But since the local store is used nothing
more than as a check point, I suppose this is not that a huge issue...

But still, I will add locking around the whole create/delete operation to prevent this scenario
from happening.

> Block Storage : store server state to persistent storage
> --------------------------------------------------------
>
>                 Key: HDFS-11154
>                 URL: https://issues.apache.org/jira/browse/HDFS-11154
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>          Components: hdfs
>            Reporter: Chen Liang
>            Assignee: Chen Liang
>         Attachments: HDFS-11154-HDFS-7240.001.patch, HDFS-11154-HDFS-7240.002.patch,
HDFS-11154-HDFS-7240.003.patch
>
>
> Currently, all the storage state are kept in server memory. If server crashes, we would
lose all the volume information. This JIRA stores server internal state into its local disk.
Such that on server failure, we can simply restart server and restore volume information from
disk.
> More specifically, the internal state written to disk is mainly the mapping from volume
to its underlying containers, plus some meta information such as volume size, block size,
etc.
> Note that this is only a simple, minimum set mechanism for persistence. It is more like
a counterpart of fsimage in HDFS, but without edit logs.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org


Mime
View raw message