Mailing-List: contact issues-help@flink.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@flink.apache.org
From: tillrohrmann <git@git.apache.org>
To: issues@flink.incubator.apache.org
Reply-To: issues@flink.incubator.apache.org
References: <git-pr-3388-flink@git.apache.org>
In-Reply-To: <git-pr-3388-flink@git.apache.org>
Subject: [GitHub] flink pull request #3388: [FLINK-5815] Add resource files configuration for ...
Content-Type: text/plain
Message-Id: <20170227130442.E8E05E0104@git1-us-west.apache.org>
Date: Mon, 27 Feb 2017 13:04:42 +0000 (UTC)
archived-at: Mon, 27 Feb 2017 13:04:57 -0000

Github user tillrohrmann commented on a diff in the pull request:

    https://github.com/apache/flink/pull/3388#discussion_r103196755
  
    --- Diff: flink-yarn/src/main/java/org/apache/flink/yarn/AbstractYarnClusterDescriptor.java ---
    @@ -742,6 +755,24 @@ public FileVisitResult preVisitDirectory(java.nio.file.Path dir, BasicFileAttrib
     			}
     		}
     
    +		for (URI originURI : archives) {
    +			Path remoteParent = Utils.getRemoteResourceRoot(appId.toString(), new Path(originURI.getPath()), fs.getHomeDirectory());
    +			String fragment = originURI.getFragment();
    +			Path target = new Path(
    +				FileUtils.localizeRemoteFiles(new org.apache.flink.core.fs.Path(remoteParent.toUri()), originURI).toUri());
    +			URI targetURI = target.toUri();
    +			if (targetURI.getFragment() == null && fragment != null) {
    +				targetURI = new URI(target.toUri().toString() + "#" + fragment);
    +			}
    +			LocalResource archive = Records.newRecord(LocalResource.class);
    +			FileStatus state = fs.getFileStatus(target);
    +			Utils.registerLocalResource(targetURI, state.getLen(), state.getModificationTime(), archive);
    +			localResources.put(fragment, archive);
    +			paths.add(new Path(targetURI));
    +			classPathBuilder.append(fragment).append(File.pathSeparator);
    +			envShipFileList.append(targetURI).append(",");
    +		}
    --- End diff --
    
    I've got the feeling that this code snippet contains a lot of path to URI and back to path conversions. Maybe we could streamline it a little bit to avoid unnecessary object instantiations.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---