hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Chen Liang (JIRA)" <j...@apache.org>
Subject [jira] [Comment Edited] (HDFS-11694) Block Storage: Add Support for 2 BlockIDBuffers and also for periodic flush of BlockIDBuffer
Date Tue, 02 May 2017 17:49:04 GMT

    [ https://issues.apache.org/jira/browse/HDFS-11694?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15993376#comment-15993376
] 

Chen Liang edited comment on HDFS-11694 at 5/2/17 5:48 PM:
-----------------------------------------------------------

Thanks [~msingh] for the patch!

A few clarification notes:
1. could you please elaborate a little bit about the reason for {{as there are maximum of
2 buffers which can be flushed at the same time.}}? Basically, what is the reasoning of *2*
BlockIDBuffers?
2. I think {{currentBuffer = acquireQueue.take();}} is a blocking call and seems there are
three places calling {{triggerBlockBufferFlush()}}. Will any of the places block infinitely?
I was mainly concerning about the {{shutdown()}} which seems possible to block forever. I
might be wrong but could you please verify these calls?

Also, this change is actually fairly complex to follow so it would be great if we can have
more comments explaining the workflow a little bit (especially {{BlockBufferManager.java}}),
otherwise it may become very difficult to maintain/modify in the future  :).


was (Author: vagarychen):
Thanks [~msingh] for the patch!

A few clarification notes:
1. could you please elaborate a little bit about the reason for {{as there are maximum of
2 buffers which can be flushed at the same time.}}?
2. I think {{currentBuffer = acquireQueue.take();}} is a blocking call and seems there are
three places calling {{triggerBlockBufferFlush()}}. Will any of the places block infinitely?
I was mainly concerning about the {{shutdown()}} which seems possible to block forever. I
might be wrong but could you please verify these calls?

Also, this change is actually fairly complex to follow so it would be great if we can have
more comments explaining the workflow a little bit (especially {{BlockBufferManager.java}}),
otherwise it may become very difficult to maintain/modify in the future  :).

> Block Storage: Add Support for 2 BlockIDBuffers and also for periodic flush of BlockIDBuffer
> --------------------------------------------------------------------------------------------
>
>                 Key: HDFS-11694
>                 URL: https://issues.apache.org/jira/browse/HDFS-11694
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>          Components: ozone
>            Reporter: Mukul Kumar Singh
>            Assignee: Mukul Kumar Singh
>         Attachments: HDFS-11694-HDFS-7240.001.patch, HDFS-11694-HDFS-7240.002.patch
>
>
> This bug tracks 2 issues
> 1) The change supports periodic flushing of BlockIdBuffer, earlier this buffer was only
flushed when either the buffer was full or shutdown was triggered
> 2) This change also adds 2 BlockIdBuffer(s), this will ensure that incoming writes are
not blocked on BlockIdBuffer flush.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org


Mime
View raw message