hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Feilong He (JIRA)" <j...@apache.org>
Subject [jira] [Comment Edited] (HDFS-14401) Refine the implementation for HDFS cache on SCM
Date Fri, 26 Apr 2019 07:57:00 GMT

    [ https://issues.apache.org/jira/browse/HDFS-14401?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16826745#comment-16826745
] 

Feilong He edited comment on HDFS-14401 at 4/26/19 7:56 AM:
------------------------------------------------------------

Thanks [~rakeshr] for your valuable comments to this work!

 
{quote}Move log message to respective constructor, that will make the FsDatasetCache.java
more cleaner.
{quote}
Yes, it would be better to move the specific log from FsDatasetCache.
{quote}How about using a {{MappableBlockLoaderFactory}} and move {{#createCacheLoader(DNConf)}} function
into that.
{quote}
Good suggestion! I will create a {{MappableBlockLoaderFactory}} to create a specific cache
loader.
{quote}Typo - '{{due to unsuccessfully mapping'}} -->to-> '{{due to unsuccessful mapping'}}.
{quote}
I will fix this log expression issue.
{quote}Can we make synchronized functions {{long release}} and {{public String getCachePath}}
{quote}
Actually, it is unnecessary to make these two functions synchronized. Their access to blockKeyToVolume
is thread safe. I made reserve() function synchronized to avoid the choosen volume is not
usable to a thread. As you know, there are two operations inside it: choose a available volume
and then reserve bytes on this volume.
{quote}{{maxBytes = pmemDir.getTotalSpace();}}, IMHO, to use [File#getUsableSpace()|https://docs.oracle.com/javase/7/docs/api/java/io/File.html#getUsableSpace()] function.
{quote}
Yes, getUsableSpace() should be used here instead of getTotalSpace().
{quote}Remove unused var in PmemVolumeManager.java - {{// private final UsedBytesCount usedBytesCount;}}
{quote}
I will remove this useless statement.
{quote}Its good to use {} instead of string concatenation in log messages. Please take care
all such occurrences in newly writing code.
{quote}
Another good suggestion! I will fix it and check other pieces of code.

 

Thanks [~rakeshr] again!


was (Author: philohe):
Thanks [~rakeshr] for your valuable comments to this work!

 
{quote}Move log message to respective constructor, that will make the FsDatasetCache.java
more cleaner.
{quote}
Yes, it would be better to move the specific log from FsDatasetCache.
{quote}How about using a {{MappableBlockLoaderFactory}} and move {{#createCacheLoader(DNConf)}} function
into that.
{quote}
Good suggestion! I will create a {{MappableBlockLoaderFactory}} to create a specific cache
loader.
{quote}Typo - '{{due to unsuccessfully mapping'}} -->to-> '{{due to unsuccessful mapping'}}.
{quote}
I will fix this log expression issue.
{quote}Can we make synchronized functions {{long release}} and {{public String getCachePath}}
{quote}
Actually, it is unnecessary to make these two functions synchronized. Their access to blockKeyToVolume
is thread safe. I made reserve() function synchronized to avoid the choosen volume is still
usable to a thread. As you know, there are two operations inside it: choose a available volume
and then reserve bytes.
{quote}{{maxBytes = pmemDir.getTotalSpace();}}, IMHO, to use [File#getUsableSpace()|https://docs.oracle.com/javase/7/docs/api/java/io/File.html#getUsableSpace()] function.
{quote}
Yes, getUsableSpace should be used here.
{quote}Remove unused var in PmemVolumeManager.java - {{// private final UsedBytesCount usedBytesCount;}}
{quote}
I will remove this useless statement.
{quote}Its good to use {} instead of string concatenation in log messages. Please take care
all such occurrences in newly writing code.
{quote}
Another good suggestion! I will fix it and check other new pieces of code.

 

Thanks [~rakeshr] again!

> Refine the implementation for HDFS cache on SCM
> -----------------------------------------------
>
>                 Key: HDFS-14401
>                 URL: https://issues.apache.org/jira/browse/HDFS-14401
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>          Components: caching, datanode
>            Reporter: Feilong He
>            Assignee: Feilong He
>            Priority: Major
>         Attachments: HDFS-14401.000.patch, HDFS-14401.001.patch, HDFS-14401.002.patch,
HDFS-14401.003.patch, HDFS-14401.004.patch, HDFS-14401.005.patch, HDFS-14401.006.patch
>
>
> In this Jira, we will refine the implementation for HDFS cache on SCM, such as: 1) Handle
full pmem volume in VolumeManager; 2) Refine pmem volume selection impl; 3) Clean up MapppableBlockLoader
interface; etc.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org


Mime
View raw message