hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Tsz Wo Nicholas Sze (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-8386) Improve synchronization of 'streamer' reference in DFSOutputStream
Date Thu, 04 Jun 2015 23:56:38 GMT

    [ https://issues.apache.org/jira/browse/HDFS-8386?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14573846#comment-14573846
] 

Tsz Wo Nicholas Sze commented on HDFS-8386:
-------------------------------------------

In trunk, the streamer synchronization bug does not exist since streamer is only initialized
in constructors.  

Also, simply replacing streamer with getStreamer() may not be really fix synchronization bugs
if there is any.  For example, in adjustChunkBoundary() below, would it work if getStreamer()
returns a different object each time?
{code}
   protected void adjustChunkBoundary() {
-    if (streamer.getAppendChunk() &&
-        streamer.getBytesCurBlock() % bytesPerChecksum == 0) {
-      streamer.setAppendChunk(false);
+    if (getStreamer().getAppendChunk() &&
+        getStreamer().getBytesCurBlock() % bytesPerChecksum == 0) {
+      getStreamer().setAppendChunk(false);
       resetChecksumBufSize();
     }
 
-    if (!streamer.getAppendChunk()) {
-      int psize = Math.min((int)(blockSize- streamer.getBytesCurBlock()),
+    if (!getStreamer().getAppendChunk()) {
+      int psize = Math.min((int)(blockSize- getStreamer().getBytesCurBlock()),
           dfsClient.getConf().getWritePacketSize());
       computePacketChunkSize(psize, bytesPerChecksum);
     }
{code}

I suggest revert the committed patch since it may potentially hide some synchronization bugs
in the HDFS-7285 branch.


> Improve synchronization of 'streamer' reference in DFSOutputStream
> ------------------------------------------------------------------
>
>                 Key: HDFS-8386
>                 URL: https://issues.apache.org/jira/browse/HDFS-8386
>             Project: Hadoop HDFS
>          Issue Type: Bug
>          Components: hdfs-client
>            Reporter: Rakesh R
>            Assignee: Rakesh R
>             Fix For: 2.8.0
>
>         Attachments: HDFS-8386-00.patch, HDFS-8386-01.patch
>
>
> Presently {{DFSOutputStream#streamer}} object reference is accessed inconsistently with
respect to synchronization. It would be good to improve this part. This has been noticed when
implementing the erasure coding feature.
> Please refer the related [discussion thread|https://issues.apache.org/jira/browse/HDFS-8294?focusedCommentId=14541411&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14541411]
in the jira HDFS-8294 for more details.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message