hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Chen Liang (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-11154) Block Storage : store server state to persistent storage
Date Mon, 21 Nov 2016 21:42:58 GMT

    [ https://issues.apache.org/jira/browse/HDFS-11154?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15684815#comment-15684815

Chen Liang commented on HDFS-11154:

Thanks [~anu] for the comments!

bq. 1. Wondering if we need a lock in CBlockManager.java between the calls.

The operations calling into {{StorageManager}} are already syncd on {{StorageManager}}'s own
lock. The next patch will include locks for those operations on the levelDB instance.

bq. 2. For function testSame, did you want to implement a Comparable interface ?

I'm under the impression that {{Comparable}} is more to support sorting a list of object,
the {{compareTo()}} returns an int value indicating an relative order, and I assumed it is
not supposed to be called directly (please correct me if I'm wrong though). Probably overriding
{{equals}} is making more sense but it requires implementing hashcode also. Since this is
only a simple method for testing, I decided not to go with a potentially confusing way. What
do you think?

bq. 3. Why are we copying the levelDB store class ? Can we move this to a place where both
Ozone and Cblock can use it instead of having 2 copies of the same class ?

Simply because I assumed the use of levelDB is only temporary and at the end of the day we
will switch to RAFT (which I suppose is the same for Ozone). Plus i was trying to not to make
any change to Ozone code base.

> Block Storage : store server state to persistent storage
> --------------------------------------------------------
>                 Key: HDFS-11154
>                 URL: https://issues.apache.org/jira/browse/HDFS-11154
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>          Components: hdfs
>            Reporter: Chen Liang
>            Assignee: Chen Liang
>         Attachments: HDFS-11154-HDFS-7240.001.patch, HDFS-11154-HDFS-7240.002.patch,
> Currently, all the storage state are kept in server memory. If server crashes, we would
lose all the volume information. This JIRA stores server internal state into its local disk.
Such that on server failure, we can simply restart server and restore volume information from
> More specifically, the internal state written to disk is mainly the mapping from volume
to its underlying containers, plus some meta information such as volume size, block size,
> Note that this is only a simple, minimum set mechanism for persistence. It is more like
a counterpart of fsimage in HDFS, but without edit logs.

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org

View raw message