hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ewan Higgs (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-12713) [READ] Refactor FileRegion and BlockAliasMap to separate out HDFS metadata and PROVIDED storage metadata
Date Wed, 08 Nov 2017 13:05:01 GMT

    [ https://issues.apache.org/jira/browse/HDFS-12713?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16243860#comment-16243860
] 

Ewan Higgs commented on HDFS-12713:
-----------------------------------

In HDFS-12665, [~virajith] suggested that we make the following changes to {{BlockAliasMap.java}}:

{code}
diff --git a/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/common/blockaliasmap/BlockAliasMap.java
b/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/common/blockaliasmap/BlockAliasMap.java
index d276fb52036..e564097fd2e 100644
--- a/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/common/blockaliasmap/BlockAliasMap.java
+++ b/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/common/blockaliasmap/BlockAliasMap.java
@@ -47,6 +47,7 @@
      */
     public abstract U resolve(Block ident) throws IOException;

+    public abstract String getBlockPoolID() throws IOException;
   }

   /**
@@ -74,10 +75,12 @@
   /**
    * Returns the writer for the alias map.
    * @param opts writer options.
+   * @param blockPoolID block pool id to use
    * @return {@link Writer} to the alias map.
    * @throws IOException
    */
-  public abstract Writer<T> getWriter(Writer.Options opts) throws IOException;
+  public abstract Writer<T> getWriter(Writer.Options opts, String blockPoolID)
+      throws IOException;
{code}

I think this is a good idea. Sketching it out in the code on my local branch I think we also
need to add BlockPoolID to the {{getReader}} call as well. This is so readers can also load
the right file, put the right bpid in the WHERE clause, etc.

> [READ] Refactor FileRegion and BlockAliasMap to separate out HDFS metadata and PROVIDED
storage metadata
> --------------------------------------------------------------------------------------------------------
>
>                 Key: HDFS-12713
>                 URL: https://issues.apache.org/jira/browse/HDFS-12713
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>            Reporter: Virajith Jalaparti
>            Assignee: Ewan Higgs
>         Attachments: HDFS-12713-HDFS-9806.001.patch
>
>




--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org


Mime
View raw message