falcon-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Mark Greene (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (FALCON-1787) Ooozie pig-action.xml requires hive sharedlib for HCatalog use
Date Wed, 27 Jan 2016 22:29:39 GMT

     [ https://issues.apache.org/jira/browse/FALCON-1787?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Mark Greene updated FALCON-1787:
--------------------------------
    Attachment: Workflow Pig Action Configuration.txt
                Workflow Job Configuration.txt
                Stack Trace.txt

Attaching stack trace of the Pig action, Falcon-generated Job configuration, falcon-generated
workflow action configuration.

> Ooozie pig-action.xml requires hive sharedlib for HCatalog use
> --------------------------------------------------------------
>
>                 Key: FALCON-1787
>                 URL: https://issues.apache.org/jira/browse/FALCON-1787
>             Project: Falcon
>          Issue Type: Bug
>          Components: oozie
>    Affects Versions: 0.6.1
>         Environment: HDP-2.3.2.0-2950
> Pig		0.15.0.2.3
> Hive	        1.2.1.2.3
> Oozie	4.2.0.2.3
> Falcon	0.6.1.2.3
>            Reporter: Mark Greene
>         Attachments: Stack Trace.txt, Workflow Job Configuration.txt, Workflow Pig Action
Configuration.txt
>
>
>  have a Pig script that I am using as the workflow for my Falcon process. The pig script
uses HCatalogStorer to write to a HCatalog URI that is the output feed defined in my Falcon
Process Entity. The Pig action in the resulting Ooozie Workflow generated by Falcon fails
with the attached stack trace. The root is that it is missing a class definitions of org/apache/hadoop/hive/shims/ShimLoader.
> Running the script manually using pig -x tex -useHCatalog <all the -params passed
by Oozie> <path to pig script> results in a successful execution. It's only once
this is called as a Pig activity in the Falcon-generated Oozie workflow that the missing class
definitions manifests.
> After some investigation I found that the Oozie workflow.xml is missing a required sharelib
decleration.
> From the workflow.xml generated by Falcon:
>                 <property>
>                     <name>oozie.action.sharelib.for.pig</name>
>                     <value>pig,hcatalog</value>
>                 </property>
> If I modify the value to include hive sharelib then the Pig action succeeds and does
not throw a missing class definition error.
> Modified workflow.xml property (works):
>     <property>
>       <name>oozie.action.sharelib.for.pig</name>
>       <value>hive,pig,hcatalog</value>
>     </property>



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message