hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Arpit Agarwal (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-8392) DataNode support for multiple datasets
Date Mon, 01 Jun 2015 22:01:17 GMT

    [ https://issues.apache.org/jira/browse/HDFS-8392?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14568109#comment-14568109

Arpit Agarwal commented on HDFS-8392:

Thanks for the review Jitendra. I fixed the first point and committed to branch-7240 since
the delta is trivial.

bq. Ideally we should change the tests to use the datasetsMap instead of legacy field 'data'.
That can be taken in a separate jira.
I'll file a Jira to make sure we fix it in trunk post merge. Deferring to avoid frequent merging

Delta for reference:
--- a/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/DataNode.java
+++ b/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/DataNode.java
@@ -2598,7 +2598,7 @@ public void scheduleAllBlockReport(long delay) {
    * @return the fsdataset that stores the blocks
-  @VisibleForTesting
+  @Deprecated
   public FsDatasetSpi<?> getFSDataset() {
     Preconditions.checkState(datasets.size() <= 1,
         "Did not expect more than one Dataset here.");
@@ -2621,7 +2621,7 @@ public BlockScanner getBlockScanner() {
    * @return
-  @VisibleForTesting
+  @Deprecated
   DirectoryScanner getDirectoryScanner() {
     return directoryScannersMap.get(getFSDataset());

> DataNode support for multiple datasets
> --------------------------------------
>                 Key: HDFS-8392
>                 URL: https://issues.apache.org/jira/browse/HDFS-8392
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>          Components: datanode
>            Reporter: Arpit Agarwal
>            Assignee: Arpit Agarwal
>         Attachments: HDFS-8392-HDFS-7240.01.patch, HDFS-8392-HDFS-7240.02.patch, HDFS-8392-HDFS-7240.03.patch
> For HDFS-7240 we would like to share available DataNode storage across HDFS blocks and
Ozone objects.
> The DataNode already supports sharing available storage across multiple block pool IDs
for the federation feature. However all federated block pools use the same dataset implementation
i.e. {{FsDatasetImpl}}.
> We can extend the DataNode to support multiple dataset implementations so the same storage
space can be shared across one or more HDFS block pools and one or more Ozone block pools.

This message was sent by Atlassian JIRA

View raw message