hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "dhruba borthakur (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HADOOP-2703) New files under lease (before close) still shows up as MISSING files/blocks in fsck
Date Fri, 11 Apr 2008 17:40:04 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-2703?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12588047#action_12588047
] 

dhruba borthakur commented on HADOOP-2703:
------------------------------------------

1. The ClientProtocol # has to be bumped because LocatedBlocks is used in a few RPCs. Please
check if NamenodeProtocol uses it too.

2. Maybe it is better to add a new field to DFSFileInfo to say whether a file is opened for
writing. The open/close characteristics is more related to a file rather than a block. This
can replace the new field that you added to LocatedBlocks.

3. The summary prints total file and total open files in two separate lines. One confusion
is whether the total file count includes the open files or not. It might be better to put
these two in the same line
     Total Files: 589 (Files that are currently being written to: 10)

4.  The "open" parameter to fsck might confuse people to wonder whether these files are opened
for reading or writing. It would be worthwhile to come up with a different name of this command
line parameter.




> New files under lease (before close) still shows up as MISSING files/blocks in fsck
> -----------------------------------------------------------------------------------
>
>                 Key: HADOOP-2703
>                 URL: https://issues.apache.org/jira/browse/HADOOP-2703
>             Project: Hadoop Core
>          Issue Type: Bug
>          Components: dfs
>    Affects Versions: 0.15.3
>            Reporter: Koji Noguchi
>            Assignee: lohit vijayarenu
>            Priority: Minor
>         Attachments: HADOOP-2703-1.patch
>
>
> On 0.15.3 which includes HADOOP-2540 ("Empty blocks make fsck report corrupt, even when
it isn't") fix, we are still seeing fsck reporting missing blocks when they aren't.   Basically,
 any unclosed small files under lease shows up as MISSING.
> Can we have an option in fsck to skip those files under lease?
> Or  count those files/blocks separately?

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message