hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Gour Saha (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-8079) YARN native service should respect source file of ConfigFile inside Service/Component spec
Date Wed, 28 Mar 2018 05:23:00 GMT

    [ https://issues.apache.org/jira/browse/YARN-8079?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16416836#comment-16416836

Gour Saha commented on YARN-8079:

I know, that's a feature Slider depends on.

Actually my bad for not making it clear. What I meant was that in YARN Service each component
has an artifact which supports 3 types today - DOCKER, TARBALL and SERVICE. I was referring
to a TARBALL type artifact and was trying to understand that if we already support this, then
will a service-owner need an ARCHIVE type config file. My counter-argument was that yes it
is needed since with TARBALL type artifact the service-owner is restricted to only one archive
file, but with the introduction of ARCHIVE type config file it opens up the gates to support
multiple archive files.

> YARN native service should respect source file of ConfigFile inside Service/Component
> ------------------------------------------------------------------------------------------
>                 Key: YARN-8079
>                 URL: https://issues.apache.org/jira/browse/YARN-8079
>             Project: Hadoop YARN
>          Issue Type: Bug
>            Reporter: Wangda Tan
>            Assignee: Wangda Tan
>            Priority: Blocker
>         Attachments: YARN-8079.001.patch, YARN-8079.002.patch, YARN-8079.003.patch
> Currently, {{srcFile}} is not respected. {{ProviderUtils}} doesn't properly read srcFile,
instead it always construct {{remoteFile}} by using componentDir and fileName of {{destFile}}:
> {code}
> Path remoteFile = new Path(compInstanceDir, fileName);
> {code} 
> To me it is a common use case which services have some files existed in HDFS and need
to be localized when components get launched. (For example, if we want to serve a Tensorflow
model, we need to localize Tensorflow model (typically not huge, less than GB) to local disk.
Otherwise launched docker container has to access HDFS.

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org

View raw message