hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Chen Liang (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-13607) [Edit Tail Fast Path Pt 1] Enhance JournalNode with an in-memory cache of recent edit transactions
Date Tue, 05 Jun 2018 16:34:00 GMT

    [ https://issues.apache.org/jira/browse/HDFS-13607?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16502063#comment-16502063
] 

Chen Liang commented on HDFS-13607:
-----------------------------------

v002 patch looks pretty good overall, just one comment. In JournaledEditsCache#storeEdits,
looks like we add the input to the buffer first, then check if it is larger then capacity,
if yes, remove the exceeding part? Seems to me that this mean dataMap can effectively go
larger than capacity. To avoid being confused by the log warn message in the future, can we
swap the order? i.e. remove first if needed, then add, something like:
{code}
while (totalSize + input.length > capacity) {
    // call dataMap.remove() and update
}
dataMap.put(...);
totalSize += input.length;
{code}

> [Edit Tail Fast Path Pt 1] Enhance JournalNode with an in-memory cache of recent edit
transactions
> --------------------------------------------------------------------------------------------------
>
>                 Key: HDFS-13607
>                 URL: https://issues.apache.org/jira/browse/HDFS-13607
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>          Components: ha, journal-node
>            Reporter: Erik Krogen
>            Assignee: Erik Krogen
>            Priority: Major
>         Attachments: HDFS-13607-HDFS-12943.000.patch, HDFS-13607-HDFS-12943.001.patch,
HDFS-13607-HDFS-12943.002.patch
>
>
> See HDFS-13150 for full design.
> This JIRA is to add the in-memory cache of recent edit transactions on the JournalNode.
This JIRA does not include accesses to this cache.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org


Mime
View raw message