flink-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (FLINK-10469) FileChannel may not write the whole buffer in a single call to FileChannel.write(Buffer buffer)
Date Wed, 03 Oct 2018 17:38:00 GMT

    [ https://issues.apache.org/jira/browse/FLINK-10469?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16637269#comment-16637269
] 

ASF GitHub Bot commented on FLINK-10469:
----------------------------------------

isunjin commented on a change in pull request #6788: [FLINK-10469][core] make sure to always
write the whole buffer to FileChannel
URL: https://github.com/apache/flink/pull/6788#discussion_r222400575
 
 

 ##########
 File path: flink-runtime/src/main/java/org/apache/flink/runtime/io/disk/iomanager/AsynchronousFileIOChannel.java
 ##########
 @@ -28,6 +28,7 @@
 import java.util.concurrent.atomic.AtomicBoolean;
 import java.util.concurrent.atomic.AtomicInteger;
 
+import static org.apache.flink.util.FileUtils.writeCompletely;
 
 Review comment:
   i would avoid import static

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


> FileChannel may not write the whole buffer in a single call to FileChannel.write(Buffer
buffer)
> -----------------------------------------------------------------------------------------------
>
>                 Key: FLINK-10469
>                 URL: https://issues.apache.org/jira/browse/FLINK-10469
>             Project: Flink
>          Issue Type: Bug
>          Components: Core, Network
>    Affects Versions: 1.4.1, 1.4.2, 1.5.3, 1.6.0, 1.6.1, 1.7.0, 1.5.4, 1.6.2
>            Reporter: Yun Gao
>            Assignee: Yun Gao
>            Priority: Blocker
>              Labels: pull-request-available
>             Fix For: 1.7.0, 1.6.2, 1.5.5
>
>
> Currently all the calls to _FileChannel.write(ByteBuffer src)_ assumes that this method
will not return before the whole buffer is written, like the one in _AsynchronousFileIOChannel.write()._
>  
> However, this assumption may not be right for all the environments. We have encountered
the case that only part of a buffer was written on a cluster with a high IO load, and the
target file got messy. 
>  
> To fix this issue, I think we should add a utility method in the org.apache.flink.util.IOUtils
to ensure the whole buffer is written with a loop,and replace all the calls to _FileChannel.write(ByteBuffer)_ with this
new method. 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Mime
View raw message