falcon-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Mark Greene <m...@markgreene.com>
Subject Re: Falcon Process fails with Pig HCatalog Usage
Date Fri, 22 Jan 2016 17:02:17 GMT
Hi Dev List,

I have tracked the issue to the *oozie.action.sharelib.for.pig* property of
the Falcon-generated workflow not reflecting the oozie.action.sharelib.for.pig
of my oozie-site.xml.

>From the workflow.xml generated by Falcon:

>From the oozie-site.xml of the cluster:

JIRA has an Issue logged but indicates it is resolved with Oozie versions
4+. My stack is running Oozie 4.2.0.

Any advice is appreciated.


On Fri, Jan 22, 2016 at 10:16 AM, Mark Greene <mark@markgreene.com> wrote:

> Hi Dev List,
> I'm trying to best understand how to troubleshoot this error (is it Oozie,
> or Falcon induced?).
> I have a Pig script that I am using as the workflow for my Falcon process.
> The pig script uses HCatalogStorer to write to a HCatalog URI that is the
> output feed defined in my Falcon Process Entity. The Pig action in the
> resulting Ooozie Workflow generated by Falcon fails with the attached stack
> trace. The root is that it is *missing a class definitions of
> org/apache/hadoop/hive/shims/ShimLoader.*
> Running the script manually using pig -x tex -useHCatalog <all the -params
> passed by Oozie> <path to pig script> results in a successful execution.
> It's only once this is called as a Pig activity in the Falcon-generated
> Oozie workflow that the missing class definitions manifests.
> I am running the following stack:
> HDP-
> Pig
> Hive
> Oozie
> Falcon
> --
> Mark Greene
> *E:* Mark@MarkGreene.com


Mark Greene
*E:* Mark@MarkGreene.com
*T: *+1 512 663 0445

  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message