hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Owen O'Malley (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HADOOP-1838) Files created with an pre-0.15 gets blocksize as zero, causing performance degradation
Date Wed, 05 Sep 2007 20:23:33 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-1838?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12525207
] 

Owen O'Malley commented on HADOOP-1838:
---------------------------------------

I'd much rather have the upgrade set the blocksize to the default block size in the case of
single block files, rather leave 0 as a special value. The problem with special values is
that they need to be tested for in every single use of the field and are thus much much harder
to maintain.

> Files created with an pre-0.15 gets blocksize as zero, causing performance degradation
> --------------------------------------------------------------------------------------
>
>                 Key: HADOOP-1838
>                 URL: https://issues.apache.org/jira/browse/HADOOP-1838
>             Project: Hadoop
>          Issue Type: Bug
>          Components: dfs
>    Affects Versions: 0.15.0
>            Reporter: dhruba borthakur
>            Assignee: dhruba borthakur
>            Priority: Blocker
>             Fix For: 0.15.0
>
>         Attachments: blockSizeZero.patch
>
>
> HADOOP-1656 introduced the support for storing block size persistently as inode metadata.
Previously, if the file has only one block then it was not possible to accurately determine
the blocksize that the application has requested at file-creation time.
> The upgrade of an older layout to the new layout kept the blocksize as zero for single-block
files that were upgraded to the new layout. This was done to indicate the DFS really does
not know the "true" blocksize of this file. This caused map-reduce to determine that a split
is 1 byte in length!

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message