hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Konstantin Shvachko (Commented) (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-1900) Use the block size key defined by common
Date Tue, 25 Oct 2011 18:14:33 GMT

    [ https://issues.apache.org/jira/browse/HDFS-1900?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13135301#comment-13135301
] 

Konstantin Shvachko commented on HDFS-1900:
-------------------------------------------

I think "dfs.block.size" introduced in HADOOP-4952 was just a mistake. I think the intention
was to have it "dfs.blocksize", same as in HDFS. I see a bunch of other file system keys in
FsConfig, that duplicate their HDFS counterparts, but the naming is consistent across them.
So I assume that extra dot between block and size was a typo.
Having said that I think it is not a good idea to duplicate key definitions. The reason is
exactly the typos or inconsistent renaming of those properties or the default values. E.g.
in common 
{{FsConfig.FS_DEFAULT_BLOCK_SIZE = 32 MB}} 
while in HDFS 
{{DFSConfigKeys.DFS_BLOCK_SIZE_DEFAULT = 64 MB}}
This is really messy.

The only method from {{FsConfig}} that is used in the code is {{getDefaultFsURI()}}. I propose
to remove everything else from {{FsConfig}} in order to avoid confusion. 
The rational behind this is that {{FsConfig}} should only contain keys that are specified
in core-site.xml. The keys that belolng to hdf-site.xml shoud be described in {{DFSConfigKeys}}.
It also looks that Tom's documentation change HDFS-671 describes keys consistently with this
assumption.
                
> Use the block size key defined by common 
> -----------------------------------------
>
>                 Key: HDFS-1900
>                 URL: https://issues.apache.org/jira/browse/HDFS-1900
>             Project: Hadoop HDFS
>          Issue Type: Improvement
>    Affects Versions: 0.21.1
>            Reporter: Eli Collins
>            Assignee: Abel Perez
>              Labels: newbie
>             Fix For: 0.22.0
>
>         Attachments: HDFS-1900.txt
>
>
> HADOOP-4952 added a dfs.block.size key to common configuration, defined in o.a.h.fs.FsConfig.
This conflicts with the original HDFS block size key of the same name, which is now deprecated
in favor of dfs.blocksize. It doesn't make sense to have two different keys for the block
size (ie they can disagree). Why doesn't HDFS just use the key defined in common?

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message