falcon-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Mark Greene <m...@markgreene.com>
Subject Re: Falcon Process fails with Pig HCatalog Usage
Date Fri, 22 Jan 2016 17:02:17 GMT
Hi Dev List,

I have tracked the issue to the *oozie.action.sharelib.for.pig* property of
the Falcon-generated workflow not reflecting the oozie.action.sharelib.for.pig
of my oozie-site.xml.

>From the workflow.xml generated by Falcon:
                <property>
                    <name>oozie.action.sharelib.for.pig</name>
                    <value>pig,hcatalog</value>
                </property>

>From the oozie-site.xml of the cluster:
    <property>
      <name>oozie.action.sharelib.for.pig</name>
      <value>hive,pig,hcatalog</value>
    </property>

JIRA has an Issue logged but indicates it is resolved with Oozie versions
4+. My stack is running Oozie 4.2.0.
https://issues.apache.org/jira/browse/FALCON-243

Any advice is appreciated.

Mark


On Fri, Jan 22, 2016 at 10:16 AM, Mark Greene <mark@markgreene.com> wrote:

> Hi Dev List,
>
> I'm trying to best understand how to troubleshoot this error (is it Oozie,
> or Falcon induced?).
>
> I have a Pig script that I am using as the workflow for my Falcon process.
> The pig script uses HCatalogStorer to write to a HCatalog URI that is the
> output feed defined in my Falcon Process Entity. The Pig action in the
> resulting Ooozie Workflow generated by Falcon fails with the attached stack
> trace. The root is that it is *missing a class definitions of
> org/apache/hadoop/hive/shims/ShimLoader.*
>
> Running the script manually using pig -x tex -useHCatalog <all the -params
> passed by Oozie> <path to pig script> results in a successful execution.
> It's only once this is called as a Pig activity in the Falcon-generated
> Oozie workflow that the missing class definitions manifests.
>
> I am running the following stack:
>
> HDP-2.3.2.0-2950
> Pig 0.15.0.2.3
> Hive 1.2.1.2.3
> Oozie 4.2.0.2.3
> Falcon 0.6.1.2.3
>
> --
>
> Mark Greene
> *E:* Mark@MarkGreene.com
>



-- 

Mark Greene
*E:* Mark@MarkGreene.com
*T: *+1 512 663 0445

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message