hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Brahma Reddy Battula (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (HDFS-13845) RBF: The default MountTableResolver should fail resolving multi-destination paths
Date Tue, 30 Oct 2018 06:06:00 GMT

     [ https://issues.apache.org/jira/browse/HDFS-13845?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Brahma Reddy Battula updated HDFS-13845:
----------------------------------------
       Resolution: Fixed
     Hadoop Flags: Reviewed
    Fix Version/s: HDFS-13891
           Status: Resolved  (was: Patch Available)

Committed to HDFS-13891 branch. [~hfyang20071] thanks for contribution.and thanks to [~elgoiri]
for additional review.

> RBF: The default MountTableResolver should fail resolving multi-destination paths
> ---------------------------------------------------------------------------------
>
>                 Key: HDFS-13845
>                 URL: https://issues.apache.org/jira/browse/HDFS-13845
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>          Components: federation, hdfs
>    Affects Versions: 3.0.0, 3.1.0, 2.9.1
>            Reporter: yanghuafeng
>            Assignee: yanghuafeng
>            Priority: Major
>             Fix For: HDFS-13891
>
>         Attachments: HDFS-13845.001.patch, HDFS-13845.002.patch, HDFS-13845.003.patch,
HDFS-13845.004.patch, HDFS-13845.005.patch
>
>
> When we use the default MountTableResolver to resolve the path, we cannot get the destination
paths for the default DestinationOrder.HASH. 
> {code:java}
> // Some comments here
> private static PathLocation buildLocation(
>       ......
>     List<RemoteLocation> locations = new LinkedList<>();
>     for (RemoteLocation oneDst : entry.getDestinations()) {
>       String nsId = oneDst.getNameserviceId();
>       String dest = oneDst.getDest();
>       String newPath = dest;
>       if (!newPath.endsWith(Path.SEPARATOR) && !remainingPath.isEmpty()) {
>         newPath += Path.SEPARATOR;
>       }
>       newPath += remainingPath;
>       RemoteLocation remoteLocation = new RemoteLocation(nsId, newPath, path);
>       locations.add(remoteLocation);
>     }
>     DestinationOrder order = entry.getDestOrder();
>     return new PathLocation(srcPath, locations, order);
>   }
> {code}
> The default order will be hash, but the HashFirstResolver will not be invoked to order
the location.
> It is ambiguous for the MountTableResolver that we will see the HASH order in the web
ui for multi-destinations path but we cannot get the result.
> In my opinion, the MountTableResolver will be a simple resolver to implement 1 to 1 not
including the 1 to n destinations. So we should check the buildLocation. If the entry has
multi destinations, we should reject it.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org


Mime
View raw message