pig-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Cheolsoo Park <piaozhe...@gmail.com>
Subject Re: [VOTE] Release Pig 0.13.0 (candidate 0)
Date Tue, 17 Jun 2014 21:56:40 GMT
I just confirmed that pig-0.12.1.tar.gz has the same issue. I've never seen
this before since I always ran e2e with the repository.

But looking at the build.xml, it seems intentional?

<exclude name="test/**/*.jar"/>

I guess we can just fix NPE.


On Tue, Jun 17, 2014 at 2:48 PM, Cheolsoo Park <piaozhexiu@gmail.com> wrote:

> Regarding e2e Native tests, they only fail in source tarball.
>
> 1) The source tarball doesn't include the following jars-
>
> test/e2e/pig/lib/hadoop-0.23.0-streaming.jar
> test/e2e/pig/lib/hadoop-examples.jar
> test/e2e/pig/lib/hadoop-streaming.jar
>
> 2) Due to #1, the following line throws NPE-
>
>
> https://github.com/apache/pig/blob/branch-0.13/src/org/apache/pig/backend/hadoop/executionengine/mapReduceLayer/JobControlCompiler.java#L1707
>
> It seems trivial to fix #2, but not sure whether #1 is a regression or not.
>
>
> On Tue, Jun 17, 2014 at 11:52 AM, Daniel Dai <daijy@hortonworks.com>
> wrote:
>
>> I will take a look of TestHBaseStorage today.
>>
>> Also run e2e tests on Hadoop 2.0.3. All clear for me.
>>
>> On Tue, Jun 17, 2014 at 11:31 AM, Rohini Palaniswamy
>> <rohini.aditya@gmail.com> wrote:
>> > I will look at the e2e test failures I reported. Can someone pick up
>> > TestHBaseStorage?
>> >
>> >
>> > On Mon, Jun 16, 2014 at 2:59 PM, Aniket Mokashi <aniket486@gmail.com>
>> wrote:
>> >
>> >> Any updates on this? Please let me know if you need me to look into
>> any of
>> >> these.
>> >>
>> >>
>> >> On Fri, Jun 13, 2014 at 9:08 AM, Cheolsoo Park <piaozhexiu@gmail.com>
>> >> wrote:
>> >>
>> >> > In my e2e on Hadoop 1 run, I found Native_1,2,3 fail. I'll file a
>> jira to
>> >> > fix them.
>> >> >
>> >> > Caused by: java.lang.NullPointerException
>> >> >     at
>> >> >
>> >> >
>> >>
>> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler.shipToHDFS(JobControlCompiler.java:1707)
>> >> >
>> >> >
>> >> > In my unit test run with jdk7, I found 4 failures. However, I only
>> see
>> >> > TestHBaseStorage fails in my jenkins build, so the rest could be due
>> to
>> >> my
>> >> > environment. Nevertheless, I'll look into them-
>> >> >
>> >> > cheolsoop@cheolsoop-gateway[502]source-tarball: grep FAILED
>> unittest.log
>> >> >     [junit] Test org.apache.pig.test.TestHBaseStorage FAILED
>> >> >     [junit] Test org.apache.pig.test.TestMRCompiler FAILED
>> >> >     [junit] Test org.apache.pig.test.TestNativeMapReduce FAILED
>> >> >     [junit] Test org.apache.pig.test.TestPigRunner FAILED
>> >> >
>> >> >
>> >> > On Thu, Jun 12, 2014 at 12:55 PM, Rohini Palaniswamy <
>> >> > rohini.aditya@gmail.com> wrote:
>> >> >
>> >> > > I ran e2e on Hadoop 0.23
>> >> > > BigData_4 failed with OOM - This certainly needs investigation
>> >> > > JobManagement_3, CmdErrors_3 failed with
>> >>  java.lang.NoClassDefFoundError:
>> >> > > org/apache/zookeeper/Shell
>> >> > >
>> >> > > Unit tests except for HBaseStorage was good. piggybank unit tests
>> are
>> >> > good.
>> >> > >
>> >> > > Regards,
>> >> > > Rohini
>> >> > >
>> >> > >
>> >> > > On Wed, Jun 11, 2014 at 1:05 PM, Daniel Dai <daijy@hortonworks.com
>> >
>> >> > wrote:
>> >> > >
>> >> > > > I will run e2e on Hadoop 2.
>> >> > > >
>> >> > > > On Wed, Jun 11, 2014 at 12:36 PM, Cheolsoo Park <
>> >> piaozhexiu@gmail.com>
>> >> > > > wrote:
>> >> > > > > I am able to run e2e on Hadoop 1 cluster.
>> >> > > > >
>> >> > > > >
>> >> > > > > On Wed, Jun 11, 2014 at 12:28 PM, Rohini Palaniswamy
<
>> >> > > > > rohini.aditya@gmail.com> wrote:
>> >> > > > >
>> >> > > > >> I have kicked off a unit test run as well. Did someone
run e2e
>> >> > tests?
>> >> > > > >>
>> >> > > > >>
>> >> > > > >> On Wed, Jun 11, 2014 at 12:26 PM, Cheolsoo Park
<
>> >> > piaozhexiu@gmail.com
>> >> > > >
>> >> > > > >> wrote:
>> >> > > > >>
>> >> > > > >> > >> Do we publish another RC?
>> >> > > > >> >
>> >> > > > >> > Unfortunately, I think we need another RC.
Can you wait
>> before
>> >> > > rolling
>> >> > > > >> out
>> >> > > > >> > a new RC until we collect as many problems
as possible? I'll
>> >> > report
>> >> > > my
>> >> > > > >> unit
>> >> > > > >> > test results by tomorrow.
>> >> > > > >> >
>> >> > > > >> > On Wed, Jun 11, 2014 at 12:09 PM, Rohini Palaniswamy
<
>> >> > > > >> > rohini.aditya@gmail.com> wrote:
>> >> > > > >> >
>> >> > > > >> > > I am also seeing that TestHBaseStorage
has 6 failures.
>> >> > > > >> > >
>> >> > > > >> > > Regards,
>> >> > > > >> > > Rohini
>> >> > > > >> > >
>> >> > > > >> > >
>> >> > > > >> > > On Wed, Jun 11, 2014 at 11:37 AM, Aniket
Mokashi <
>> >> > > > aniket486@gmail.com>
>> >> > > > >> > > wrote:
>> >> > > > >> > >
>> >> > > > >> > > > Thanks Cheolsoo, thanks for catching
this.
>> >> > > > >> > > >
>> >> > > > >> > > > Do you know what's the best way of
fixing this? Do we
>> >> publish
>> >> > > > another
>> >> > > > >> > RC?
>> >> > > > >> > > > I will make sure to clean working
directory before I
>> package
>> >> > > next
>> >> > > > >> time.
>> >> > > > >> > > >
>> >> > > > >> > > > Thanks,
>> >> > > > >> > > > Aniket
>> >> > > > >> > > >
>> >> > > > >> > > >
>> >> > > > >> > > > On Wed, Jun 11, 2014 at 8:38 AM,
Cheolsoo Park <
>> >> > > > piaozhexiu@gmail.com
>> >> > > > >> >
>> >> > > > >> > > > wrote:
>> >> > > > >> > > >
>> >> > > > >> > > > > Hi Aniket,
>> >> > > > >> > > > >
>> >> > > > >> > > > > Thank you so much for the hard
work!
>> >> > > > >> > > > >
>> >> > > > >> > > > > I discovered that several temporary
files are
>> included in
>> >> > the
>> >> > > > >> source
>> >> > > > >> > > > > tarball. We should remove them
from the tarball.
>> >> > > > >> > > > >
>> >> > > > >> > > > > Here is the diff between the
source tarball and the
>> >> > > repository-
>> >> > > > >> > > > >
>> >> > > > >> > > > > Only in repo/contrib/piggybank/java/lib:
.gitignore
>> >> > > > >> > > > > Only in source-tarball/contrib/piggybank/java:
>> >> > > > >> > > > > testPathPartitioner-testGetKeys-1402332960101
>> >> > > > >> > > > > Only in repo: .eclipse.templates
>> >> > > > >> > > > > Only in repo: .git
>> >> > > > >> > > > > Only in repo: .gitignore
>> >> > > > >> > > > > Only in source-tarball: script3.txt
>> >> > > > >> > > > > Only in
>> >> > > source-tarball/src/docs/src/documentation/content/xdocs:
>> >> > > > >> > > > > func.xml.orig
>> >> > > > >> > > > > Only in repo/test/e2e/pig/lib:
>> hadoop-0.23.0-streaming.jar
>> >> > > > >> > > > > Only in repo/test/e2e/pig/lib:
hadoop-examples.jar
>> >> > > > >> > > > > Only in repo/test/e2e/pig/lib:
hadoop-streaming.jar
>> >> > > > >> > > > > Only in
>> >> > source-tarball/test/org/apache/pig/builtin/avro/data:
>> >> > > > avro
>> >> > > > >> > > > > Only in
>> >> > > > source-tarball/test/org/apache/pig/builtin/avro/data/json:
>> >> > > > >> > > > > evenFileNameTestDirectoryCounts.json
>> >> > > > >> > > > > Only in
>> >> > > > source-tarball/test/org/apache/pig/builtin/avro/data/json:
>> >> > > > >> > > > > simpleRecordsTrevni.json
>> >> > > > >> > > > > Only in
>> >> > > > source-tarball/test/org/apache/pig/builtin/avro/data/json:
>> >> > > > >> > > > > testdirectory
>> >> > > > >> > > > > Only in
>> >> > > > source-tarball/test/org/apache/pig/builtin/avro/data/json:
>> >> > > > >> > > > > testDirectoryCounts.json
>> >> > > > >> > > > > Only in
>> >> > source-tarball/test/org/apache/pig/builtin/avro/data:
>> >> > > > >> trevni
>> >> > > > >> > > > > Only in repo/test/org/apache/pig/test/data:
>> >> > pigtestloader.jar
>> >> > > > >> > > > > Only in repo/test/org/apache/pig/test/data:
>> >> > TestWordCount.jar
>> >> > > > >> > > > > Only in repo/test/perf/pigmix/lib:
sdsuLibJKD12.jar
>> >> > > > >> > > > > Only in source-tarball:
>> >> > > > >> > > > >
>> >> > > > >> > >
>> >> > > > >>
>> >> > >
>> >>
>> TestDataBagAccess-testBagStoreLoad-org.apache.pig.builtin.BinStorage.txt
>> >> > > > >> > > > > Only in source-tarball:
>> >> > > > >> > > > >
>> >> > > > >> > >
>> >> > > > >>
>> >> > >
>> >>
>> TestDataBagAccess-testBagStoreLoad-org.apache.pig.builtin.PigStorage.txt
>> >> > > > >> > > > >
>> >> > > > >> > > > > Thanks,
>> >> > > > >> > > > > Cheolsoo
>> >> > > > >> > > > >
>> >> > > > >> > > > >
>> >> > > > >> > > > > On Mon, Jun 9, 2014 at 11:59
AM, Aniket Mokashi <
>> >> > > > >> aniket486@gmail.com
>> >> > > > >> > >
>> >> > > > >> > > > > wrote:
>> >> > > > >> > > > >
>> >> > > > >> > > > > > Hi,
>> >> > > > >> > > > > >
>> >> > > > >> > > > > > I have created a candidate
build for Pig 0.13.0.
>> >> > > > >> > > > > >
>> >> > > > >> > > > > > Keys used to sign the release
are available at
>> >> > > > >> > > > > >
>> http://svn.apache.org/viewvc/pig/trunk/KEYS?view=markup
>> >> .
>> >> > > > >> > > > > >
>> >> > > > >> > > > > > Please download, test,
and try it out:
>> >> > > > >> > > > > > http://people.apache.org/~aniket486/pig-0.13.0-rc0
>> >> > > > >> > > > > >
>> >> > > > >> > > > > > Release notes and the rat
report are available at
>> the
>> >> same
>> >> > > > >> > location.
>> >> > > > >> > > > > >
>> >> > > > >> > > > > > Should we release this?
Vote closes on Tuesday, June
>> >> 17th
>> >> > > > 2014.
>> >> > > > >> > > > > >
>> >> > > > >> > > > > > Thanks,
>> >> > > > >> > > > > > Aniket
>> >> > > > >> > > > > >
>> >> > > > >> > > > >
>> >> > > > >> > > >
>> >> > > > >> > > >
>> >> > > > >> > > >
>> >> > > > >> > > > --
>> >> > > > >> > > > "...:::Aniket:::... Quetzalco@tl"
>> >> > > > >> > > >
>> >> > > > >> > >
>> >> > > > >> >
>> >> > > > >>
>> >> > > >
>> >> > > > --
>> >> > > > CONFIDENTIALITY NOTICE
>> >> > > > NOTICE: This message is intended for the use of the individual
or
>> >> > entity
>> >> > > to
>> >> > > > which it is addressed and may contain information that is
>> >> confidential,
>> >> > > > privileged and exempt from disclosure under applicable law.
If
>> the
>> >> > reader
>> >> > > > of this message is not the intended recipient, you are hereby
>> >> notified
>> >> > > that
>> >> > > > any printing, copying, dissemination, distribution, disclosure
or
>> >> > > > forwarding of this communication is strictly prohibited.
If you
>> have
>> >> > > > received this communication in error, please contact the
sender
>> >> > > immediately
>> >> > > > and delete it from your system. Thank You.
>> >> > > >
>> >> > >
>> >> >
>> >>
>> >>
>> >>
>> >> --
>> >> "...:::Aniket:::... Quetzalco@tl"
>> >>
>>
>> --
>> CONFIDENTIALITY NOTICE
>> NOTICE: This message is intended for the use of the individual or entity
>> to
>> which it is addressed and may contain information that is confidential,
>> privileged and exempt from disclosure under applicable law. If the reader
>> of this message is not the intended recipient, you are hereby notified
>> that
>> any printing, copying, dissemination, distribution, disclosure or
>> forwarding of this communication is strictly prohibited. If you have
>> received this communication in error, please contact the sender
>> immediately
>> and delete it from your system. Thank You.
>>
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message