hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hudson (Commented) (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-362) FSEditLog should not writes long and short as UTF8 and should not use ArrayWritable for writing non-array items
Date Mon, 16 Jan 2012 22:59:38 GMT

    [ https://issues.apache.org/jira/browse/HDFS-362?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13187283#comment-13187283
] 

Hudson commented on HDFS-362:
-----------------------------

Integrated in Hadoop-Common-0.23-Commit #384 (See [https://builds.apache.org/job/Hadoop-Common-0.23-Commit/384/])
    HDFS-362. svn merge -c 1171945 from trunk

eli : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1232203
Files : 
* /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
* /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/protocol/LayoutVersion.java
* /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSEditLogOp.java
* /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSImageSerialization.java
* /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/tools/offlineEditsViewer/EditsLoaderCurrent.java
* /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/tools/offlineImageViewer/ImageLoaderCurrent.java
* /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestFSEditLogLoader.java

                
> FSEditLog should not writes long and short as UTF8 and should not use ArrayWritable for
writing non-array items
> ---------------------------------------------------------------------------------------------------------------
>
>                 Key: HDFS-362
>                 URL: https://issues.apache.org/jira/browse/HDFS-362
>             Project: Hadoop HDFS
>          Issue Type: Improvement
>          Components: name-node
>    Affects Versions: 0.23.0, 0.24.0
>            Reporter: Tsz Wo (Nicholas), SZE
>            Assignee: Uma Maheswara Rao G
>             Fix For: 0.23.1
>
>         Attachments: HDFS-362.1.patch, HDFS-362.2.patch, HDFS-362.2b.patch, HDFS-362.2c.patch,
HDFS-362.2d.patch, HDFS-362.2d.patch, HDFS-362.2e.patch, HDFS-362.patch
>
>
> In FSEditLog, 
> - long and short are first converted to String and are further converted to UTF8
> - For some non-array items, it first create an ArrayWritable object to hold all the items
and then writes the ArrayWritable object.
> These result creating many intermediate objects which affects Namenode CPU performance
and Namenode restart.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message