pig-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Daniel Dai <da...@hortonworks.com>
Subject Re: [VOTE] Release Pig 0.13.0 (candidate 0)
Date Tue, 17 Jun 2014 22:13:31 GMT
Oh, probably it is intentional. Alan, do you know if we shall include
test jars into the source release?

On Tue, Jun 17, 2014 at 2:56 PM, Cheolsoo Park <piaozhexiu@gmail.com> wrote:
> I just confirmed that pig-0.12.1.tar.gz has the same issue. I've never seen
> this before since I always ran e2e with the repository.
>
> But looking at the build.xml, it seems intentional?
>
> <exclude name="test/**/*.jar"/>
>
> I guess we can just fix NPE.
>
>
> On Tue, Jun 17, 2014 at 2:48 PM, Cheolsoo Park <piaozhexiu@gmail.com> wrote:
>
>> Regarding e2e Native tests, they only fail in source tarball.
>>
>> 1) The source tarball doesn't include the following jars-
>>
>> test/e2e/pig/lib/hadoop-0.23.0-streaming.jar
>> test/e2e/pig/lib/hadoop-examples.jar
>> test/e2e/pig/lib/hadoop-streaming.jar
>>
>> 2) Due to #1, the following line throws NPE-
>>
>>
>> https://github.com/apache/pig/blob/branch-0.13/src/org/apache/pig/backend/hadoop/executionengine/mapReduceLayer/JobControlCompiler.java#L1707
>>
>> It seems trivial to fix #2, but not sure whether #1 is a regression or not.
>>
>>
>> On Tue, Jun 17, 2014 at 11:52 AM, Daniel Dai <daijy@hortonworks.com>
>> wrote:
>>
>>> I will take a look of TestHBaseStorage today.
>>>
>>> Also run e2e tests on Hadoop 2.0.3. All clear for me.
>>>
>>> On Tue, Jun 17, 2014 at 11:31 AM, Rohini Palaniswamy
>>> <rohini.aditya@gmail.com> wrote:
>>> > I will look at the e2e test failures I reported. Can someone pick up
>>> > TestHBaseStorage?
>>> >
>>> >
>>> > On Mon, Jun 16, 2014 at 2:59 PM, Aniket Mokashi <aniket486@gmail.com>
>>> wrote:
>>> >
>>> >> Any updates on this? Please let me know if you need me to look into
>>> any of
>>> >> these.
>>> >>
>>> >>
>>> >> On Fri, Jun 13, 2014 at 9:08 AM, Cheolsoo Park <piaozhexiu@gmail.com>
>>> >> wrote:
>>> >>
>>> >> > In my e2e on Hadoop 1 run, I found Native_1,2,3 fail. I'll file
a
>>> jira to
>>> >> > fix them.
>>> >> >
>>> >> > Caused by: java.lang.NullPointerException
>>> >> >     at
>>> >> >
>>> >> >
>>> >>
>>> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler.shipToHDFS(JobControlCompiler.java:1707)
>>> >> >
>>> >> >
>>> >> > In my unit test run with jdk7, I found 4 failures. However, I only
>>> see
>>> >> > TestHBaseStorage fails in my jenkins build, so the rest could be
due
>>> to
>>> >> my
>>> >> > environment. Nevertheless, I'll look into them-
>>> >> >
>>> >> > cheolsoop@cheolsoop-gateway[502]source-tarball: grep FAILED
>>> unittest.log
>>> >> >     [junit] Test org.apache.pig.test.TestHBaseStorage FAILED
>>> >> >     [junit] Test org.apache.pig.test.TestMRCompiler FAILED
>>> >> >     [junit] Test org.apache.pig.test.TestNativeMapReduce FAILED
>>> >> >     [junit] Test org.apache.pig.test.TestPigRunner FAILED
>>> >> >
>>> >> >
>>> >> > On Thu, Jun 12, 2014 at 12:55 PM, Rohini Palaniswamy <
>>> >> > rohini.aditya@gmail.com> wrote:
>>> >> >
>>> >> > > I ran e2e on Hadoop 0.23
>>> >> > > BigData_4 failed with OOM - This certainly needs investigation
>>> >> > > JobManagement_3, CmdErrors_3 failed with
>>> >>  java.lang.NoClassDefFoundError:
>>> >> > > org/apache/zookeeper/Shell
>>> >> > >
>>> >> > > Unit tests except for HBaseStorage was good. piggybank unit
tests
>>> are
>>> >> > good.
>>> >> > >
>>> >> > > Regards,
>>> >> > > Rohini
>>> >> > >
>>> >> > >
>>> >> > > On Wed, Jun 11, 2014 at 1:05 PM, Daniel Dai <daijy@hortonworks.com
>>> >
>>> >> > wrote:
>>> >> > >
>>> >> > > > I will run e2e on Hadoop 2.
>>> >> > > >
>>> >> > > > On Wed, Jun 11, 2014 at 12:36 PM, Cheolsoo Park <
>>> >> piaozhexiu@gmail.com>
>>> >> > > > wrote:
>>> >> > > > > I am able to run e2e on Hadoop 1 cluster.
>>> >> > > > >
>>> >> > > > >
>>> >> > > > > On Wed, Jun 11, 2014 at 12:28 PM, Rohini Palaniswamy
<
>>> >> > > > > rohini.aditya@gmail.com> wrote:
>>> >> > > > >
>>> >> > > > >> I have kicked off a unit test run as well. Did
someone run e2e
>>> >> > tests?
>>> >> > > > >>
>>> >> > > > >>
>>> >> > > > >> On Wed, Jun 11, 2014 at 12:26 PM, Cheolsoo Park
<
>>> >> > piaozhexiu@gmail.com
>>> >> > > >
>>> >> > > > >> wrote:
>>> >> > > > >>
>>> >> > > > >> > >> Do we publish another RC?
>>> >> > > > >> >
>>> >> > > > >> > Unfortunately, I think we need another
RC. Can you wait
>>> before
>>> >> > > rolling
>>> >> > > > >> out
>>> >> > > > >> > a new RC until we collect as many problems
as possible? I'll
>>> >> > report
>>> >> > > my
>>> >> > > > >> unit
>>> >> > > > >> > test results by tomorrow.
>>> >> > > > >> >
>>> >> > > > >> > On Wed, Jun 11, 2014 at 12:09 PM, Rohini
Palaniswamy <
>>> >> > > > >> > rohini.aditya@gmail.com> wrote:
>>> >> > > > >> >
>>> >> > > > >> > > I am also seeing that TestHBaseStorage
has 6 failures.
>>> >> > > > >> > >
>>> >> > > > >> > > Regards,
>>> >> > > > >> > > Rohini
>>> >> > > > >> > >
>>> >> > > > >> > >
>>> >> > > > >> > > On Wed, Jun 11, 2014 at 11:37 AM,
Aniket Mokashi <
>>> >> > > > aniket486@gmail.com>
>>> >> > > > >> > > wrote:
>>> >> > > > >> > >
>>> >> > > > >> > > > Thanks Cheolsoo, thanks for catching
this.
>>> >> > > > >> > > >
>>> >> > > > >> > > > Do you know what's the best way
of fixing this? Do we
>>> >> publish
>>> >> > > > another
>>> >> > > > >> > RC?
>>> >> > > > >> > > > I will make sure to clean working
directory before I
>>> package
>>> >> > > next
>>> >> > > > >> time.
>>> >> > > > >> > > >
>>> >> > > > >> > > > Thanks,
>>> >> > > > >> > > > Aniket
>>> >> > > > >> > > >
>>> >> > > > >> > > >
>>> >> > > > >> > > > On Wed, Jun 11, 2014 at 8:38
AM, Cheolsoo Park <
>>> >> > > > piaozhexiu@gmail.com
>>> >> > > > >> >
>>> >> > > > >> > > > wrote:
>>> >> > > > >> > > >
>>> >> > > > >> > > > > Hi Aniket,
>>> >> > > > >> > > > >
>>> >> > > > >> > > > > Thank you so much for the
hard work!
>>> >> > > > >> > > > >
>>> >> > > > >> > > > > I discovered that several
temporary files are
>>> included in
>>> >> > the
>>> >> > > > >> source
>>> >> > > > >> > > > > tarball. We should remove
them from the tarball.
>>> >> > > > >> > > > >
>>> >> > > > >> > > > > Here is the diff between
the source tarball and the
>>> >> > > repository-
>>> >> > > > >> > > > >
>>> >> > > > >> > > > > Only in repo/contrib/piggybank/java/lib:
.gitignore
>>> >> > > > >> > > > > Only in source-tarball/contrib/piggybank/java:
>>> >> > > > >> > > > > testPathPartitioner-testGetKeys-1402332960101
>>> >> > > > >> > > > > Only in repo: .eclipse.templates
>>> >> > > > >> > > > > Only in repo: .git
>>> >> > > > >> > > > > Only in repo: .gitignore
>>> >> > > > >> > > > > Only in source-tarball:
script3.txt
>>> >> > > > >> > > > > Only in
>>> >> > > source-tarball/src/docs/src/documentation/content/xdocs:
>>> >> > > > >> > > > > func.xml.orig
>>> >> > > > >> > > > > Only in repo/test/e2e/pig/lib:
>>> hadoop-0.23.0-streaming.jar
>>> >> > > > >> > > > > Only in repo/test/e2e/pig/lib:
hadoop-examples.jar
>>> >> > > > >> > > > > Only in repo/test/e2e/pig/lib:
hadoop-streaming.jar
>>> >> > > > >> > > > > Only in
>>> >> > source-tarball/test/org/apache/pig/builtin/avro/data:
>>> >> > > > avro
>>> >> > > > >> > > > > Only in
>>> >> > > > source-tarball/test/org/apache/pig/builtin/avro/data/json:
>>> >> > > > >> > > > > evenFileNameTestDirectoryCounts.json
>>> >> > > > >> > > > > Only in
>>> >> > > > source-tarball/test/org/apache/pig/builtin/avro/data/json:
>>> >> > > > >> > > > > simpleRecordsTrevni.json
>>> >> > > > >> > > > > Only in
>>> >> > > > source-tarball/test/org/apache/pig/builtin/avro/data/json:
>>> >> > > > >> > > > > testdirectory
>>> >> > > > >> > > > > Only in
>>> >> > > > source-tarball/test/org/apache/pig/builtin/avro/data/json:
>>> >> > > > >> > > > > testDirectoryCounts.json
>>> >> > > > >> > > > > Only in
>>> >> > source-tarball/test/org/apache/pig/builtin/avro/data:
>>> >> > > > >> trevni
>>> >> > > > >> > > > > Only in repo/test/org/apache/pig/test/data:
>>> >> > pigtestloader.jar
>>> >> > > > >> > > > > Only in repo/test/org/apache/pig/test/data:
>>> >> > TestWordCount.jar
>>> >> > > > >> > > > > Only in repo/test/perf/pigmix/lib:
sdsuLibJKD12.jar
>>> >> > > > >> > > > > Only in source-tarball:
>>> >> > > > >> > > > >
>>> >> > > > >> > >
>>> >> > > > >>
>>> >> > >
>>> >>
>>> TestDataBagAccess-testBagStoreLoad-org.apache.pig.builtin.BinStorage.txt
>>> >> > > > >> > > > > Only in source-tarball:
>>> >> > > > >> > > > >
>>> >> > > > >> > >
>>> >> > > > >>
>>> >> > >
>>> >>
>>> TestDataBagAccess-testBagStoreLoad-org.apache.pig.builtin.PigStorage.txt
>>> >> > > > >> > > > >
>>> >> > > > >> > > > > Thanks,
>>> >> > > > >> > > > > Cheolsoo
>>> >> > > > >> > > > >
>>> >> > > > >> > > > >
>>> >> > > > >> > > > > On Mon, Jun 9, 2014 at 11:59
AM, Aniket Mokashi <
>>> >> > > > >> aniket486@gmail.com
>>> >> > > > >> > >
>>> >> > > > >> > > > > wrote:
>>> >> > > > >> > > > >
>>> >> > > > >> > > > > > Hi,
>>> >> > > > >> > > > > >
>>> >> > > > >> > > > > > I have created a candidate
build for Pig 0.13.0.
>>> >> > > > >> > > > > >
>>> >> > > > >> > > > > > Keys used to sign the
release are available at
>>> >> > > > >> > > > > >
>>> http://svn.apache.org/viewvc/pig/trunk/KEYS?view=markup
>>> >> .
>>> >> > > > >> > > > > >
>>> >> > > > >> > > > > > Please download, test,
and try it out:
>>> >> > > > >> > > > > > http://people.apache.org/~aniket486/pig-0.13.0-rc0
>>> >> > > > >> > > > > >
>>> >> > > > >> > > > > > Release notes and the
rat report are available at
>>> the
>>> >> same
>>> >> > > > >> > location.
>>> >> > > > >> > > > > >
>>> >> > > > >> > > > > > Should we release this?
Vote closes on Tuesday, June
>>> >> 17th
>>> >> > > > 2014.
>>> >> > > > >> > > > > >
>>> >> > > > >> > > > > > Thanks,
>>> >> > > > >> > > > > > Aniket
>>> >> > > > >> > > > > >
>>> >> > > > >> > > > >
>>> >> > > > >> > > >
>>> >> > > > >> > > >
>>> >> > > > >> > > >
>>> >> > > > >> > > > --
>>> >> > > > >> > > > "...:::Aniket:::... Quetzalco@tl"
>>> >> > > > >> > > >
>>> >> > > > >> > >
>>> >> > > > >> >
>>> >> > > > >>
>>> >> > > >
>>> >> > > > --
>>> >> > > > CONFIDENTIALITY NOTICE
>>> >> > > > NOTICE: This message is intended for the use of the individual
or
>>> >> > entity
>>> >> > > to
>>> >> > > > which it is addressed and may contain information that
is
>>> >> confidential,
>>> >> > > > privileged and exempt from disclosure under applicable
law. If
>>> the
>>> >> > reader
>>> >> > > > of this message is not the intended recipient, you are
hereby
>>> >> notified
>>> >> > > that
>>> >> > > > any printing, copying, dissemination, distribution, disclosure
or
>>> >> > > > forwarding of this communication is strictly prohibited.
If you
>>> have
>>> >> > > > received this communication in error, please contact
the sender
>>> >> > > immediately
>>> >> > > > and delete it from your system. Thank You.
>>> >> > > >
>>> >> > >
>>> >> >
>>> >>
>>> >>
>>> >>
>>> >> --
>>> >> "...:::Aniket:::... Quetzalco@tl"
>>> >>
>>>
>>> --
>>> CONFIDENTIALITY NOTICE
>>> NOTICE: This message is intended for the use of the individual or entity
>>> to
>>> which it is addressed and may contain information that is confidential,
>>> privileged and exempt from disclosure under applicable law. If the reader
>>> of this message is not the intended recipient, you are hereby notified
>>> that
>>> any printing, copying, dissemination, distribution, disclosure or
>>> forwarding of this communication is strictly prohibited. If you have
>>> received this communication in error, please contact the sender
>>> immediately
>>> and delete it from your system. Thank You.
>>>
>>
>>

-- 
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to 
which it is addressed and may contain information that is confidential, 
privileged and exempt from disclosure under applicable law. If the reader 
of this message is not the intended recipient, you are hereby notified that 
any printing, copying, dissemination, distribution, disclosure or 
forwarding of this communication is strictly prohibited. If you have 
received this communication in error, please contact the sender immediately 
and delete it from your system. Thank You.

Mime
View raw message