hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Konstantin Shvachko (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HADOOP-807) splitting DFS version number
Date Sat, 09 Dec 2006 02:26:22 GMT
    [ http://issues.apache.org/jira/browse/HADOOP-807?page=comments#action_12457042 ] 
            
Konstantin Shvachko commented on HADOOP-807:
--------------------------------------------

Unified storage version ensures that all system components are in sync.

We wanted all system components to run the same software version.
The reason is that we have seen cases (before versions where introduced)
when some data-nodes where not shutdown properly and caused different kinds of problems.
Using a unified storage version somewhat relaxes the same software requirement.
If we want to further relax this requirement we need a clearly defined versioning story.
Which might not be trivial. There are different approaches, here are just some

http://www.cs.cmu.edu/~srini/Papers/publications/dead/usenix05-distapp/usenix05.pdf
http://publications.csail.mit.edu/tmp/MIT-CSAIL-TR-2005-078.pdf



> splitting DFS version number
> ----------------------------
>
>                 Key: HADOOP-807
>                 URL: http://issues.apache.org/jira/browse/HADOOP-807
>             Project: Hadoop
>          Issue Type: Bug
>          Components: dfs
>            Reporter: dhruba borthakur
>         Assigned To: dhruba borthakur
>
> In the current code, there is a enum named DFS_CURRENT_VERSION. It is being used for
three different purposes:
> 1. It is stored in the fsimage and represents the format of the fsimage.
> 2. It is stored in the "storage" file by the Datanodes. Thus, it represents the format
of data on Datanodes.
> 3. It is used as a version of the Datanode Protocol.
> The current implementation makes it difficult to change the fsimage format without affecting
the Datanode Protocol. My proposal is to introduce three new constants, one for each of the
functionality described above.
> In the current code, we have:
>     public static final int DFS_CURRENT_VERSION = -3;
> Instead, we can have:
>     public static final int DFS_FSIMAGE_CURRENT_VERSION = -3;
>     public static final int DFS_DATANODEPROTOCOL_CURRENT_VERSION = -3;
>     public static final int DFS_DATASTORAGE_CURRENT_VERSION = -3;
> The first one is associated with the format of the fsimage/edits; the second one is associated
with the communication procotol between the namenode and the datanode; the third one is associated
with the format of the datastore on the datanodes (e.g. directory fanout).
> Please comment.

-- 
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message