hive-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Thejas M Nair (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HIVE-10066) Hive on Tez job submission through WebHCat doesn't ship Tez artifacts
Date Thu, 26 Mar 2015 22:14:53 GMT

    [ https://issues.apache.org/jira/browse/HIVE-10066?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14382800#comment-14382800
] 

Thejas M Nair commented on HIVE-10066:
--------------------------------------

[~ekoifman] Some questions about the objectives, based on the config param description - 

bq. If the path /foo/bar is a directory, the contents of the the entire dir will be localized
 and ./foo/* will be added to HADOOP_CLASSPATH
Is it /foo/bar that is localized or is it /foo ? If it is /foo/bar, why is /foo/* added to
classpath ?

bq. In the example above, "./tez-site.xml:./tez-client/*:./lib/*" will be added to  HADOOP_CLASSPATH.
Or is it "./tez-site.xml:./tez-client/*:/tez-client/lib/*"


> Hive on Tez job submission through WebHCat doesn't ship Tez artifacts
> ---------------------------------------------------------------------
>
>                 Key: HIVE-10066
>                 URL: https://issues.apache.org/jira/browse/HIVE-10066
>             Project: Hive
>          Issue Type: Bug
>          Components: Tez, WebHCat
>    Affects Versions: 1.0.0
>            Reporter: Eugene Koifman
>            Assignee: Eugene Koifman
>         Attachments: HIVE-10066.patch
>
>
> From [~hitesh]:
> "Tez is a client-side only component ( no daemons, etc ) and therefore it is meant to
be installed on the gateway box ( or where its client libraries are needed by any other services’
daemons). It does not have any cluster dependencies both in terms of libraries/jars as well
as configs. When it runs on a worker node, everything was pre-packaged and made available
to the worker node via the distributed cache via the client code. Hence, its client-side configs
are also only needed on the same (client) node as where it is installed. The only other install
step needed is to have the tez tarball be uploaded to HDFS and the config has an entry “tez.lib.uris”
which points to the HDFS path. "
> We need a way to pass client jars and tez-site.xml to the LaunchMapper.
> We should create a general purpose mechanism here which can supply additional artifacts
per job type.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message