hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Sverre Bakke (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (HDFS-7941) hsync() not working
Date Tue, 31 Mar 2015 14:47:06 GMT

     [ https://issues.apache.org/jira/browse/HDFS-7941?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel

Sverre Bakke updated HDFS-7941:
    Summary: hsync() not working  (was: SequenceFile.Writer.hsync() not working?)

> hsync() not working
> -------------------
>                 Key: HDFS-7941
>                 URL: https://issues.apache.org/jira/browse/HDFS-7941
>             Project: Hadoop HDFS
>          Issue Type: Bug
>          Components: hdfs-client
>    Affects Versions: 2.6.0
>         Environment: HDP 2.2 running on Redhat
>            Reporter: Sverre Bakke
> When using SequenceFile.Writer and appending+syncing to file repeatedly, the sync does
not appear to work other than:
> - once after writing headers
> - when closing.
> Imagine the following test case:
> http://pastebin.com/Y9xysCRX
> This code would append a new record every second and then immediately sync it. One would
also imagine that the file would grow for every append, however, this does not happen.
> After watching the behavior I have noticed that it only syncs the headers at the very
beginning (providing a file of 164 bytes) and then never again until its closed. This despite
it is asked to hsync() after every append.
> Looking into the debug logs, this also claims the same behavior (executed the provided
code example and grepped for "sync"):
> SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
> SLF4J: Defaulting to no-operation (NOP) logger implementation
> SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details.
> 2015-03-17 15:55:14 DEBUG ProtobufRpcEngine:253 - Call: fsync took 11ms
> This was the only time the code ran fsync throughout the entire execution.

This message was sent by Atlassian JIRA

View raw message