Mailing-List: contact hadoop-dev-help@lucene.apache.org; run by ezmlm
Precedence: bulk
Reply-To: hadoop-dev@lucene.apache.org
Message-ID: <3474844.1186696782782.JavaMail.jira@brutus>
Date: Thu, 9 Aug 2007 14:59:42 -0700 (PDT)
From: "Konstantin Shvachko (JIRA)" <jira@apache.org>
To: hadoop-dev@lucene.apache.org
Subject: [jira] Commented: (HADOOP-89) files are not visible until they are
 closed
In-Reply-To: <1845748515.1142561833009.JavaMail.jira@ajax>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit


    [ https://issues.apache.org/jira/browse/HADOOP-89?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12518853 ] 

Konstantin Shvachko commented on HADOOP-89:
-------------------------------------------

I see 2 problems with this patch:
# It does not close any of the 3 issues raised in the description above.
You can see files in the ls, but you cannot see file progress and you still loose all accumulated data if the file is not closed.
# Implementation-wise I would expect that when files are made visible we will get rid of the pending create structure, 
but it is still there, so now you will have to synchronize pending create data with the corresponding inode.

I'd propose to make files both visible and readable while they are still being created. Namely, we should let
clients read portions of the file that have been written and replicated on data-nodes.

> files are not visible until they are closed
> -------------------------------------------
>
>                 Key: HADOOP-89
>                 URL: https://issues.apache.org/jira/browse/HADOOP-89
>             Project: Hadoop
>          Issue Type: Bug
>          Components: dfs
>    Affects Versions: 0.1.0
>            Reporter: Yoram Arnon
>            Assignee: dhruba borthakur
>            Priority: Critical
>         Attachments: atomicCreation.patch
>
>
> the current behaviour, whereby a file is not visible until it is closed has several flaws,including:
> 1. no practical way to know if a file/job is progressing
> 2. no way to implement files that never close, such as log files
> 3. failure to close a file results in loss of the file
> The part of the file that's written should be visible.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.