Mailing-List: contact dev-help@pig.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@pig.apache.org
Received-SPF: neutral (nike.apache.org: local policy)
MIME-Version: 1.0
In-Reply-To: 
 <CAOehgTnFcSkE49y5GgXx4v6L_5YQQbSeOWSTvfOdWO3CzOTYcw@mail.gmail.com>
References: 
 <CAB2zpW8s+NLBsD1E3vzRRQpk8B8pGm4RcFiWPprd6VDREsaJ=Q@mail.gmail.com>
	<CAOehgT=paRT_r5H2H_wYr4=zQZ=ipWf65pdDY4+mw=toh9orRg@mail.gmail.com>
	<CAB2zpW-mLfc_5bSQzCs0DV4swD0HgROOfb73VYRsPcSj9POnNQ@mail.gmail.com>
	<CAE7pYjZ7g8=fW56KrOw=VY3BrxA7SoxCdpK6Fiy+MJAJ5bbADA@mail.gmail.com>
	<CAE7pYjaUwoNS+F2mXHo=H3rgJ06G5mNrDYAj5TUh=fBVRZOpMA@mail.gmail.com>
	<CAOehgTn50+Z0V+pttiO=6Knq66ujm9C=jNX=Y8igMAh_mV2UnQ@mail.gmail.com>
	<CAE7pYjbnBTMrjCDsg9jK5_mWndBjkM4TJQ-D8bC_HjT1mCs4PA@mail.gmail.com>
	<CAOehgTnFcSkE49y5GgXx4v6L_5YQQbSeOWSTvfOdWO3CzOTYcw@mail.gmail.com>
Date: Tue, 24 Apr 2012 00:09:41 -0700
Message-ID: 
 <CAB2zpW__wgknzkBTgh4C32Z0gM9M05TGUK0dk_7bROT_FKDVmA@mail.gmail.com>
Subject: Re: [VOTE] Release Pig 0.10.0 (candidate 0)
From: Daniel Dai <daijy@hortonworks.com>
To: dev@pig.apache.org
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable

Do you have HADOOP_HOME? Both HADOOP_CONF_DIR/PIG_CLASSPATH should
work, can you use bin/pig -secretDebugCmd to check hadoop command
line?

On Mon, Apr 23, 2012 at 11:32 PM, Prashant Kommireddi
<prash1784@gmail.com> wrote:
> Thanks Dmitriy, that works. But I am wondering why the behavior is
> different from the previous versions.
>
> Difference I see in bin/pig is (0.10.0 vs 0.9.1)
>
>> # add HADOOP_CONF_DIR
>> if [ "$HADOOP_CONF_DIR" !=3D "" ]; then
>> =A0 =A0 CLASSPATH=3D${CLASSPATH}:${HADOOP_CONF_DIR}
>> fi
>
> AFAIK, this should not affect it - all it's doing is adding the conf dir =
to
> the classpath which I was doing earlier through PIG_CLASSPATH in my wrapp=
er
> script.
>
> The issue here is that certain properties are not same between client
> machine and remote cluster, for eg JAVA_HOME. Since pig is client side it
> made sense for Pig to not pick up any cluster properties from
> "hadoop-env.sh". I am not sure what the change here is that's now causing
> it to be picked up.
>
>
>
> On Mon, Apr 23, 2012 at 9:14 PM, Dmitriy Ryaboy <dvryaboy@gmail.com> wrot=
e:
>
>> pig.sh understands the arguments below -- try setting HADOOP_CONF_DIR?
>>
>> # Environment Variables
>> #
>> # =A0 =A0 JAVA_HOME =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0The java implementati=
on to use.
>> Overrides JAVA_HOME.
>> #
>> # =A0 =A0 PIG_CLASSPATH Extra Java CLASSPATH entries.
>> #
>> # =A0 =A0 HADOOP_HOME/HADOOP_PREFIX =A0 =A0 Environment
>> HADOOP_HOME/HADOOP_PREFIX(0.20.205)
>> #
>> # =A0 =A0 HADOOP_CONF_DIR =A0 =A0 Hadoop conf dir
>> #
>> # =A0 =A0 PIG_HEAPSIZE =A0 =A0The maximum amount of heap to use, in MB.
>> # =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =
=A0 =A0 =A0Default is 1000.
>> #
>> # =A0 =A0 PIG_OPTS =A0 =A0 =A0 =A0 =A0 =A0Extra Java runtime options.
>> #
>> # =A0 =A0 PIG_CONF_DIR =A0 =A0Alternate conf dir. Default is ${PIG_HOME}=
/conf.
>> #
>> # =A0 =A0 HBASE_CONF_DIR - Optionally, the HBase configuration to run ag=
ainst
>> # =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0when using HBaseStorage
>>
>>
>>
>> On Mon, Apr 23, 2012 at 8:45 PM, Prashant Kommireddi
>> <prash1784@gmail.com> wrote:
>> > I have a wrapper script to switch between Pig versions and clusters.
>> >
>> > export PIG_HOME=3D$HOME/tools/Linux/hadoop/pig-$PIG_VERSION
>> > export JAVA_HOME=3D$HOME/tools/Linux/jdk/jdk$JAVA_VERSION/
>> > export
>> PIG_CLASSPATH=3D$HOME/apps/gridforce/main/hadoop/conf/$HADOOP_CLUSTER
>> >
>> > HADOOP_CLUSTER contains the hadoop configs (endpoints) for the cluster=
 I
>> > want to point to.
>> >
>> > And then I do this to start pig.
>> >
>> > $PIG_HOME/bin/pig
>> >
>> > This works with previous versions pig-0.8.0 and pig-0.9.1. However,
>> > pig-0.10.0 fails to pick up the right classpath.
>> >
>> > $ ./pig.sh
>> > =A0..
>> > .....
>> > ......
>> > 2012-04-23 20:42:35,340 [main] INFO
>> > org.apache.pig.backend.hadoop.executionengine.HExecutionEngine -
>> Connecting
>> > to hadoop file system at: file:///
>> >
>> > Is this something with my script, or may be the new version (0.10.0)?
>> >
>> > -Prashant
>> >
>> >
>> > On Mon, Apr 23, 2012 at 8:30 PM, Dmitriy Ryaboy <dvryaboy@gmail.com>
>> wrote:
>> >
>> >> Uh, actually, one of the test-commit tests failed in my environment.
>> >>
>> >> In TestPigServer:
>> >>
>> >> Testcase: testDefaultPigProperties took 0.033 sec
>> >> =A0 =A0 =A0 =A0Caused an ERROR
>> >> null
>> >> java.lang.NullPointerException
>> >> =A0 =A0 =A0 =A0at
>> >>
>> org.apache.pig.test.TestPigServer.testDefaultPigProperties(TestPigServer=
.java:895)
>> >>
>> >> Something about my environment?
>> >>
>> >> D
>> >>
>> >>
>> >> On Mon, Apr 23, 2012 at 6:36 PM, Dmitriy Ryaboy <dvryaboy@gmail.com>
>> >> wrote:
>> >> > +1
>> >> >
>> >> >
>> >> > Verified several jobs using Elephant-Bird loaders.
>> >> > Tested correctness with pig.exec.mapPartAgg both true and false.
>> >> > Verified license.
>> >> > Verified release notes.
>> >> > Ran test-commit
>> >> >
>> >> > D
>> >> >
>> >> > On Sat, Apr 21, 2012 at 12:27 PM, Daniel Dai <daijy@hortonworks.com=
>
>> >> wrote:
>> >> >> We should do sanity check of the package, such as unit tests, e2e
>> >> >> tests, piggybank tests, package integrity, package signature,
>> license,
>> >> >> etc. However, if we find a new bug, usually we will push it to the
>> >> >> next release at this stage unless it is a critical one.
>> >> >>
>> >> >> Thanks,
>> >> >> Daniel
>> >> >>
>> >> >> On Sat, Apr 21, 2012 at 12:48 AM, Prashant Kommireddi
>> >> >> <prash1784@gmail.com> wrote:
>> >> >>> Hi Daniel,
>> >> >>>
>> >> >>> What is required other than running the regular tests for testing
>> >> release
>> >> >>> candidate? I can think of running a few existing scripts against
>> >> candidate
>> >> >>> build and making sure outputs look fine.
>> >> >>>
>> >> >>> Thanks,
>> >> >>> Prashant
>> >> >>>
>> >> >>> On Fri, Apr 20, 2012 at 12:39 AM, Daniel Dai <daijy@hortonworks.c=
om
>> >
>> >> wrote:
>> >> >>>
>> >> >>>> Hi,
>> >> >>>>
>> >> >>>> I have created a candidate build for Pig 0.10.0.
>> >> >>>>
>> >> >>>> Keys used to sign the release are available at
>> >> >>>> http://svn.apache.org/viewvc/pig/trunk/KEYS?view=3Dmarkup.
>> >> >>>>
>> >> >>>> Please download, test, and try it out:
>> >> >>>>
>> >> >>>> http://people.apache.org/~daijy/pig-0.10.0-candidate-0/
>> >> >>>>
>> >> >>>> Should we release this? Vote closes on next Tuesday, Apr 24th.
>> >> >>>>
>> >> >>>> Daniel
>> >> >>>>
>> >>
>>