hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Bharath Mundlapudi (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-2092) Remove configuration object reference in DFSClient
Date Thu, 23 Jun 2011 23:54:47 GMT

    [ https://issues.apache.org/jira/browse/HDFS-2092?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13054184#comment-13054184
] 

Bharath Mundlapudi commented on HDFS-2092:
------------------------------------------

I am going to create umbrella jira for this. Link these Jira's to that soon. Thanks for asking.

Regarding how much we save in memory is a context dependent meaning clients usage. From my
past studies on this shows the following distribution.

Cluster1: 

Quantile (Size in Bytes):
        1%        50%        90%        95%        99%       100% 
   20958.0    27277.0   115043.4   150415.7   273566.7 82323144.0 

Cluster2:

Quantile (Size in Bytes):
      1%       50%       90%       95%       99%      100% 
  21087.0   46540.0  153724.0  240260.0  386204.8 7885728.0 



 

> Remove configuration object reference in DFSClient
> --------------------------------------------------
>
>                 Key: HDFS-2092
>                 URL: https://issues.apache.org/jira/browse/HDFS-2092
>             Project: Hadoop HDFS
>          Issue Type: Bug
>          Components: hdfs client
>    Affects Versions: 0.23.0
>            Reporter: Bharath Mundlapudi
>            Assignee: Bharath Mundlapudi
>             Fix For: 0.23.0
>
>         Attachments: HDFS-2092-1.patch, HDFS-2092-2.patch
>
>
> At present, DFSClient stores reference to configuration object. Since, these configuration
objects are pretty big at times can blot the processes which has multiple DFSClient objects
like in TaskTracker. This is an attempt to remove the reference of conf object in DFSClient.


--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message