ambari-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Andrii Tkach (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (AMBARI-10129) Pig Client needs Tez Client on same host, ambari web need to update HostRoles it sends
Date Thu, 19 Mar 2015 13:28:38 GMT

     [ https://issues.apache.org/jira/browse/AMBARI-10129?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Andrii Tkach updated AMBARI-10129:
----------------------------------
    Attachment: AMBARI-10129.patch

> Pig Client needs Tez Client on same host, ambari web need to update HostRoles it sends
> --------------------------------------------------------------------------------------
>
>                 Key: AMBARI-10129
>                 URL: https://issues.apache.org/jira/browse/AMBARI-10129
>             Project: Ambari
>          Issue Type: Bug
>          Components: ambari-web
>    Affects Versions: 2.0.0
>            Reporter: Andrii Tkach
>            Assignee: Andrii Tkach
>             Fix For: 2.1.0
>
>         Attachments: AMBARI-10129.patch, Screen Shot 2015-02-11 at 12.13.11 PM.png, Screen
Shot 2015-02-11 at 12.15.16 PM.png, Screen Shot 2015-02-11 at 12.23.56 PM.png
>
>
> Pig service check fails if the Tez client is not installed on the same host that contains
the Pig client.
> If during the Install Wizard, Pig and Tez are added at the same time, then this guarantees
that whatever hosts are selected to contain the clients will have both services, which works.
> However, if Tez is added first (plus it dependencies on MR2, Yarn) on say host 3, and
then Pig service is added, and the client is only installed on host 1, then the Pig service
check will fail.
> I tried adding this to PIG's metainfo.xml
> {code}
> <dependencies>
>   <dependency>
>     <name>TEZ/TEZ_CLIENT</name>
>     <scope>host</scope>
>     <auto-deploy>
>       <enabled>true</enabled>
>     </auto-deploy>
>   </dependency>
>   <dependency>
>     <name>MAPREDUCE2/MAPREDUCE2_CLIENT</name>
>     <scope>host</scope>
>     <auto-deploy>
>       <enabled>true</enabled>
>     </auto-deploy>
>   </dependency>
> </dependencies>
> {code}
> However, it was not sufficient for Tez Client to be installed on the same hosts.
> When I installed Pig, I saw that ambari-web only sent the request as,
> {code}
> POST http://c6401.ambari.apache.org:8080/api/v1/clusters/c1/hosts
> {"RequestInfo":{"query":"Hosts/host_name:
> c6401.ambari.apache.org"},"Body":{"host_components":[{"HostRoles":{"component_name":"PIG"}}]}}
> {code}
> *Notice that the HostRoles only contains PIG; it should also be adding TEZ_CLIENT and
MAPREDUCE2_CLIENT, as per the dependencies shown above*
> This is the failure that Pig's service check generates,
> {code}
> 15/02/11 20:17:28 INFO pig.ExecTypeProvider: Picked TEZ as the ExecType
> ...
> 2015-02-11 20:17:35,966 [PigTezLauncher-0] ERROR org.apache.pig.backend.hadoop.executionengine.tez.TezJob
- Cannot submit DAG
> org.apache.tez.dag.api.TezUncheckedException: Invalid configuration of tez jars, tez.lib.uris
is not defined in the configuration
> 	at org.apache.tez.client.TezClientUtils.setupTezJarsLocalResources(TezClientUtils.java:162)
> 	at org.apache.tez.client.TezClient.getTezJarResources(TezClient.java:721)
> 	at org.apache.tez.client.TezClient.start(TezClient.java:298)
> 	at org.apache.pig.backend.hadoop.executionengine.tez.TezSessionManager.createSession(TezSessionManager.java:95)
> 	at org.apache.pig.backend.hadoop.executionengine.tez.TezSessionManager.getClient(TezSessionManager.java:195)
> 	at org.apache.pig.backend.hadoop.executionengine.tez.TezJob.run(TezJob.java:159)
> 	at org.apache.pig.backend.hadoop.executionengine.tez.TezLauncher$1.run(TezLauncher.java:167)
> 	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
> 	at java.util.concurrent.FutureTask.run(FutureTask.java:262)
> 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> 	at java.lang.Thread.run(Thread.java:745)
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message