hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jing Zhao (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-5738) Serialize INode information in protobuf
Date Mon, 13 Jan 2014 18:34:53 GMT

    [ https://issues.apache.org/jira/browse/HDFS-5738?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13869803#comment-13869803
] 

Jing Zhao commented on HDFS-5738:
---------------------------------

# The current 002 patch still cannot be compiled. Looks like you are missing the following
changes in the patch:
{code}
diff --git a/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSImageFormatProtobuf.java
b/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/n
index 344a6a0..18dd768 100644
--- a/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSImageFormatProtobuf.java
+++ b/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSImageFormatProtobuf.java
@@ -195,6 +195,7 @@ private void loadINode(InputStream in, FileHeader.Section header)
   static final class Saver {
     final SaveNamespaceContext context;
     private MD5Hash savedDigest;
+    private long currentOffset = PRE_ALLOCATED_HEADER_SIZE;
 
     Saver(SaveNamespaceContext context) {
       this.context = context;
diff --git a/hadoop-hdfs-project/hadoop-hdfs/src/main/proto/fsimage.proto b/hadoop-hdfs-project/hadoop-hdfs/src/main/proto/fsimage.proto
index 5df8fd1..0855102 100644
--- a/hadoop-hdfs-project/hadoop-hdfs/src/main/proto/fsimage.proto
+++ b/hadoop-hdfs-project/hadoop-hdfs/src/main/proto/fsimage.proto
@@ -55,6 +55,7 @@ message FileHeader {
   message Section {
     optional string name = 1;
     optional uint64 length = 2;
+    optional uint64 offset = 3;
   }
   repeated Section sections = 5;
 }
{code}
# In the meanwhile, for non-snapshot information, we also need to handle FileUnderConstruction
information. This can be handled in either this jira or a separate jira (such as HDFS-5743).
# The section index information may be moved to the end of the fsimge as a footer? This can
simplify the current code and avoid the 1KB allocation. This is optional and we can continue
improving the protobuf definition in new jiras.


> Serialize INode information in protobuf
> ---------------------------------------
>
>                 Key: HDFS-5738
>                 URL: https://issues.apache.org/jira/browse/HDFS-5738
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>            Reporter: Haohui Mai
>            Assignee: Haohui Mai
>         Attachments: HDFS-5738.000.patch, HDFS-5738.001.patch, HDFS-5738.002.patch
>
>
> This jira proposes to serialize inode information with protobuf. Snapshot-related information
are out of the scope of this jira.



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

Mime
View raw message