hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Robert Joseph Evans (Commented) (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-989) Flush/sync do not work on Hadoop LocalFileSystem
Date Mon, 07 Nov 2011 14:54:52 GMT

    [ https://issues.apache.org/jira/browse/HDFS-989?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13145536#comment-13145536
] 

Robert Joseph Evans commented on HDFS-989:
------------------------------------------

Does HDFS have a similar issue with flush (Not flushing everything unless we are at a byte
boundary)?  It being a checksum file system as well.  If so then I think we also need to update
the documentation about when data is flushed and perhaps even provide an API so that a user
can know how many bytes they may need to pad before the flush can happen as expected.  From
reading the documentation on trunk for Syncable it does not indicate at all that this might
happen.

{code}
  /**
   * @deprecated As of HADOOP 0.21.0, replaced by hflush
   * @see #hflush()
   */
  @Deprecated  public void sync() throws IOException;

  /** Flush out the data in client's user buffer. After the return of
   * this call, new readers will see the data.
   * @throws IOException if any error occurs
   */
  public void hflush() throws IOException;

  /** Similar to posix fsync, flush out the data in client's user buffer 
   * all the way to the disk device (but the disk may have it in its cache).
   * @throws IOException if error occurs
   */
  public void hsync() throws IOException;
{code}
                
> Flush/sync do not work on Hadoop LocalFileSystem
> ------------------------------------------------
>
>                 Key: HDFS-989
>                 URL: https://issues.apache.org/jira/browse/HDFS-989
>             Project: Hadoop HDFS
>          Issue Type: Bug
>    Affects Versions: 0.20.1
>            Reporter: Nathan Marz
>
> They seem to be no-ops. This is really easy to reproduce, just open a file using FileSystem/getLocal(new
Configuration()), write data to the output stream, and then try to flush/sync. I also tried
creating the output stream with a buffer size of 1, but that had no effect.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message