bigtop-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jay Vyas <jayunit...@gmail.com>
Subject Re: pig maven integration
Date Sat, 26 Apr 2014 20:02:06 GMT
Hi S:

I would suggest you look into apache bigtop's bigpetstore project and
borrow from the pig profile in the pom.xml file there.  it does essentially
what you want, and also has all the pig libraries necesssary for running
the whole thing in a maven task.

For an intro to the bigpetstore project's goals you can watch the youtube
demo: https://www.youtube.com/watch?v=OVB3nEKN94k, which also shows how we
test the pig portion locally and then how we run the same thing in a
cluster.

To test it locally, you do : mvn clean verify -P pig.  You can easily adopt
the TestPig*IT.java class for your own integration testing needs.

To run the same thing on the cluster, you run the corresponding pig class
in a hadoop job.

See
https://github.com/apache/bigtop/blob/master/bigtop-bigpetstore/README.mdfor
details .


On Sat, Apr 26, 2014 at 2:01 PM, Salabhanjika S <salabhanjika9@gmail.com>wrote:

> Hi,
>
> I'm new to Pig scripting. Please provide me some pointers on the following.
>
> 1. How can we just *compile (compile only)* pig scripts? I had gone through
> the documentation and *"-check"* is providing the syntax check
> functionality. But it requires all params used in the script to be set. I'm
> looking for something more generic. So that I can have a quick sanity check
> of my scripts.
>
> 2. Also, what is the clean way to handle library dependencies of a Pig
> script? Current way of registering the jars by path looks very odd to me.
> This requires changes in script/code when there is a library upgrade.
>
>
> -S
>



-- 
Jay Vyas
http://jayunit100.blogspot.com

Mime
View raw message