Return-Path: X-Original-To: apmail-pig-dev-archive@www.apache.org Delivered-To: apmail-pig-dev-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 6E8801183D for ; Tue, 17 Jun 2014 21:57:09 +0000 (UTC) Received: (qmail 99871 invoked by uid 500); 17 Jun 2014 21:57:09 -0000 Delivered-To: apmail-pig-dev-archive@pig.apache.org Received: (qmail 99767 invoked by uid 500); 17 Jun 2014 21:57:09 -0000 Mailing-List: contact dev-help@pig.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@pig.apache.org Delivered-To: mailing list dev@pig.apache.org Received: (qmail 99722 invoked by uid 99); 17 Jun 2014 21:57:08 -0000 Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 17 Jun 2014 21:57:08 +0000 X-ASF-Spam-Status: No, hits=2.5 required=5.0 tests=FREEMAIL_REPLY,HTML_MESSAGE,RCVD_IN_DNSWL_LOW,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (nike.apache.org: domain of piaozhexiu@gmail.com designates 209.85.223.176 as permitted sender) Received: from [209.85.223.176] (HELO mail-ie0-f176.google.com) (209.85.223.176) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 17 Jun 2014 21:57:05 +0000 Received: by mail-ie0-f176.google.com with SMTP id rd18so6643882iec.7 for ; Tue, 17 Jun 2014 14:56:41 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type; bh=vO2NFYYKt94bmXJQsH5M6iC9Qq7uOY4gyU76y7nFcek=; b=0DpIyYHr0kJrZ3SdFRhlZgd5ycM2qqfUp0KyU0P1ktT5tk/V4JBT39RDvEm+zDquZg ON5mtHW3/BoR2VC+Tf9x96lYkPBqeRrPv9kgQGRhDPMvmwiEhuZAtf4iqPcM6iz4lnQR xOQ9M40zwQ9CBAGkg4APqRfumMVmMr9z/5ot7uDiXsbp9QxxUQJAonTwKdzof73cOSg/ 44RouSUJBZuMQxoT6055pP3RuU8WU3YEwA/I8iWJ1yKZ9kZIdhblZgd5PQLw9lw5DZr5 6iUfBTmM/0PTaa9qGHpok2Xy8eDBocy7BwDTGLcjzMwlnXDJ7WKxhiro+8txN1BphSBz MKXQ== MIME-Version: 1.0 X-Received: by 10.42.111.135 with SMTP id u7mr5250801icp.80.1403042200755; Tue, 17 Jun 2014 14:56:40 -0700 (PDT) Received: by 10.64.102.136 with HTTP; Tue, 17 Jun 2014 14:56:40 -0700 (PDT) In-Reply-To: References: Date: Tue, 17 Jun 2014 14:56:40 -0700 Message-ID: Subject: Re: [VOTE] Release Pig 0.13.0 (candidate 0) From: Cheolsoo Park To: "dev@pig.apache.org" Content-Type: multipart/alternative; boundary=001a11c3b956fb643a04fc0f371d X-Virus-Checked: Checked by ClamAV on apache.org --001a11c3b956fb643a04fc0f371d Content-Type: text/plain; charset=UTF-8 I just confirmed that pig-0.12.1.tar.gz has the same issue. I've never seen this before since I always ran e2e with the repository. But looking at the build.xml, it seems intentional? I guess we can just fix NPE. On Tue, Jun 17, 2014 at 2:48 PM, Cheolsoo Park wrote: > Regarding e2e Native tests, they only fail in source tarball. > > 1) The source tarball doesn't include the following jars- > > test/e2e/pig/lib/hadoop-0.23.0-streaming.jar > test/e2e/pig/lib/hadoop-examples.jar > test/e2e/pig/lib/hadoop-streaming.jar > > 2) Due to #1, the following line throws NPE- > > > https://github.com/apache/pig/blob/branch-0.13/src/org/apache/pig/backend/hadoop/executionengine/mapReduceLayer/JobControlCompiler.java#L1707 > > It seems trivial to fix #2, but not sure whether #1 is a regression or not. > > > On Tue, Jun 17, 2014 at 11:52 AM, Daniel Dai > wrote: > >> I will take a look of TestHBaseStorage today. >> >> Also run e2e tests on Hadoop 2.0.3. All clear for me. >> >> On Tue, Jun 17, 2014 at 11:31 AM, Rohini Palaniswamy >> wrote: >> > I will look at the e2e test failures I reported. Can someone pick up >> > TestHBaseStorage? >> > >> > >> > On Mon, Jun 16, 2014 at 2:59 PM, Aniket Mokashi >> wrote: >> > >> >> Any updates on this? Please let me know if you need me to look into >> any of >> >> these. >> >> >> >> >> >> On Fri, Jun 13, 2014 at 9:08 AM, Cheolsoo Park >> >> wrote: >> >> >> >> > In my e2e on Hadoop 1 run, I found Native_1,2,3 fail. I'll file a >> jira to >> >> > fix them. >> >> > >> >> > Caused by: java.lang.NullPointerException >> >> > at >> >> > >> >> > >> >> >> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler.shipToHDFS(JobControlCompiler.java:1707) >> >> > >> >> > >> >> > In my unit test run with jdk7, I found 4 failures. However, I only >> see >> >> > TestHBaseStorage fails in my jenkins build, so the rest could be due >> to >> >> my >> >> > environment. Nevertheless, I'll look into them- >> >> > >> >> > cheolsoop@cheolsoop-gateway[502]source-tarball: grep FAILED >> unittest.log >> >> > [junit] Test org.apache.pig.test.TestHBaseStorage FAILED >> >> > [junit] Test org.apache.pig.test.TestMRCompiler FAILED >> >> > [junit] Test org.apache.pig.test.TestNativeMapReduce FAILED >> >> > [junit] Test org.apache.pig.test.TestPigRunner FAILED >> >> > >> >> > >> >> > On Thu, Jun 12, 2014 at 12:55 PM, Rohini Palaniswamy < >> >> > rohini.aditya@gmail.com> wrote: >> >> > >> >> > > I ran e2e on Hadoop 0.23 >> >> > > BigData_4 failed with OOM - This certainly needs investigation >> >> > > JobManagement_3, CmdErrors_3 failed with >> >> java.lang.NoClassDefFoundError: >> >> > > org/apache/zookeeper/Shell >> >> > > >> >> > > Unit tests except for HBaseStorage was good. piggybank unit tests >> are >> >> > good. >> >> > > >> >> > > Regards, >> >> > > Rohini >> >> > > >> >> > > >> >> > > On Wed, Jun 11, 2014 at 1:05 PM, Daniel Dai > > >> >> > wrote: >> >> > > >> >> > > > I will run e2e on Hadoop 2. >> >> > > > >> >> > > > On Wed, Jun 11, 2014 at 12:36 PM, Cheolsoo Park < >> >> piaozhexiu@gmail.com> >> >> > > > wrote: >> >> > > > > I am able to run e2e on Hadoop 1 cluster. >> >> > > > > >> >> > > > > >> >> > > > > On Wed, Jun 11, 2014 at 12:28 PM, Rohini Palaniswamy < >> >> > > > > rohini.aditya@gmail.com> wrote: >> >> > > > > >> >> > > > >> I have kicked off a unit test run as well. Did someone run e2e >> >> > tests? >> >> > > > >> >> >> > > > >> >> >> > > > >> On Wed, Jun 11, 2014 at 12:26 PM, Cheolsoo Park < >> >> > piaozhexiu@gmail.com >> >> > > > >> >> > > > >> wrote: >> >> > > > >> >> >> > > > >> > >> Do we publish another RC? >> >> > > > >> > >> >> > > > >> > Unfortunately, I think we need another RC. Can you wait >> before >> >> > > rolling >> >> > > > >> out >> >> > > > >> > a new RC until we collect as many problems as possible? I'll >> >> > report >> >> > > my >> >> > > > >> unit >> >> > > > >> > test results by tomorrow. >> >> > > > >> > >> >> > > > >> > On Wed, Jun 11, 2014 at 12:09 PM, Rohini Palaniswamy < >> >> > > > >> > rohini.aditya@gmail.com> wrote: >> >> > > > >> > >> >> > > > >> > > I am also seeing that TestHBaseStorage has 6 failures. >> >> > > > >> > > >> >> > > > >> > > Regards, >> >> > > > >> > > Rohini >> >> > > > >> > > >> >> > > > >> > > >> >> > > > >> > > On Wed, Jun 11, 2014 at 11:37 AM, Aniket Mokashi < >> >> > > > aniket486@gmail.com> >> >> > > > >> > > wrote: >> >> > > > >> > > >> >> > > > >> > > > Thanks Cheolsoo, thanks for catching this. >> >> > > > >> > > > >> >> > > > >> > > > Do you know what's the best way of fixing this? Do we >> >> publish >> >> > > > another >> >> > > > >> > RC? >> >> > > > >> > > > I will make sure to clean working directory before I >> package >> >> > > next >> >> > > > >> time. >> >> > > > >> > > > >> >> > > > >> > > > Thanks, >> >> > > > >> > > > Aniket >> >> > > > >> > > > >> >> > > > >> > > > >> >> > > > >> > > > On Wed, Jun 11, 2014 at 8:38 AM, Cheolsoo Park < >> >> > > > piaozhexiu@gmail.com >> >> > > > >> > >> >> > > > >> > > > wrote: >> >> > > > >> > > > >> >> > > > >> > > > > Hi Aniket, >> >> > > > >> > > > > >> >> > > > >> > > > > Thank you so much for the hard work! >> >> > > > >> > > > > >> >> > > > >> > > > > I discovered that several temporary files are >> included in >> >> > the >> >> > > > >> source >> >> > > > >> > > > > tarball. We should remove them from the tarball. >> >> > > > >> > > > > >> >> > > > >> > > > > Here is the diff between the source tarball and the >> >> > > repository- >> >> > > > >> > > > > >> >> > > > >> > > > > Only in repo/contrib/piggybank/java/lib: .gitignore >> >> > > > >> > > > > Only in source-tarball/contrib/piggybank/java: >> >> > > > >> > > > > testPathPartitioner-testGetKeys-1402332960101 >> >> > > > >> > > > > Only in repo: .eclipse.templates >> >> > > > >> > > > > Only in repo: .git >> >> > > > >> > > > > Only in repo: .gitignore >> >> > > > >> > > > > Only in source-tarball: script3.txt >> >> > > > >> > > > > Only in >> >> > > source-tarball/src/docs/src/documentation/content/xdocs: >> >> > > > >> > > > > func.xml.orig >> >> > > > >> > > > > Only in repo/test/e2e/pig/lib: >> hadoop-0.23.0-streaming.jar >> >> > > > >> > > > > Only in repo/test/e2e/pig/lib: hadoop-examples.jar >> >> > > > >> > > > > Only in repo/test/e2e/pig/lib: hadoop-streaming.jar >> >> > > > >> > > > > Only in >> >> > source-tarball/test/org/apache/pig/builtin/avro/data: >> >> > > > avro >> >> > > > >> > > > > Only in >> >> > > > source-tarball/test/org/apache/pig/builtin/avro/data/json: >> >> > > > >> > > > > evenFileNameTestDirectoryCounts.json >> >> > > > >> > > > > Only in >> >> > > > source-tarball/test/org/apache/pig/builtin/avro/data/json: >> >> > > > >> > > > > simpleRecordsTrevni.json >> >> > > > >> > > > > Only in >> >> > > > source-tarball/test/org/apache/pig/builtin/avro/data/json: >> >> > > > >> > > > > testdirectory >> >> > > > >> > > > > Only in >> >> > > > source-tarball/test/org/apache/pig/builtin/avro/data/json: >> >> > > > >> > > > > testDirectoryCounts.json >> >> > > > >> > > > > Only in >> >> > source-tarball/test/org/apache/pig/builtin/avro/data: >> >> > > > >> trevni >> >> > > > >> > > > > Only in repo/test/org/apache/pig/test/data: >> >> > pigtestloader.jar >> >> > > > >> > > > > Only in repo/test/org/apache/pig/test/data: >> >> > TestWordCount.jar >> >> > > > >> > > > > Only in repo/test/perf/pigmix/lib: sdsuLibJKD12.jar >> >> > > > >> > > > > Only in source-tarball: >> >> > > > >> > > > > >> >> > > > >> > > >> >> > > > >> >> >> > > >> >> >> TestDataBagAccess-testBagStoreLoad-org.apache.pig.builtin.BinStorage.txt >> >> > > > >> > > > > Only in source-tarball: >> >> > > > >> > > > > >> >> > > > >> > > >> >> > > > >> >> >> > > >> >> >> TestDataBagAccess-testBagStoreLoad-org.apache.pig.builtin.PigStorage.txt >> >> > > > >> > > > > >> >> > > > >> > > > > Thanks, >> >> > > > >> > > > > Cheolsoo >> >> > > > >> > > > > >> >> > > > >> > > > > >> >> > > > >> > > > > On Mon, Jun 9, 2014 at 11:59 AM, Aniket Mokashi < >> >> > > > >> aniket486@gmail.com >> >> > > > >> > > >> >> > > > >> > > > > wrote: >> >> > > > >> > > > > >> >> > > > >> > > > > > Hi, >> >> > > > >> > > > > > >> >> > > > >> > > > > > I have created a candidate build for Pig 0.13.0. >> >> > > > >> > > > > > >> >> > > > >> > > > > > Keys used to sign the release are available at >> >> > > > >> > > > > > >> http://svn.apache.org/viewvc/pig/trunk/KEYS?view=markup >> >> . >> >> > > > >> > > > > > >> >> > > > >> > > > > > Please download, test, and try it out: >> >> > > > >> > > > > > http://people.apache.org/~aniket486/pig-0.13.0-rc0 >> >> > > > >> > > > > > >> >> > > > >> > > > > > Release notes and the rat report are available at >> the >> >> same >> >> > > > >> > location. >> >> > > > >> > > > > > >> >> > > > >> > > > > > Should we release this? Vote closes on Tuesday, June >> >> 17th >> >> > > > 2014. >> >> > > > >> > > > > > >> >> > > > >> > > > > > Thanks, >> >> > > > >> > > > > > Aniket >> >> > > > >> > > > > > >> >> > > > >> > > > > >> >> > > > >> > > > >> >> > > > >> > > > >> >> > > > >> > > > >> >> > > > >> > > > -- >> >> > > > >> > > > "...:::Aniket:::... Quetzalco@tl" >> >> > > > >> > > > >> >> > > > >> > > >> >> > > > >> > >> >> > > > >> >> >> > > > >> >> > > > -- >> >> > > > CONFIDENTIALITY NOTICE >> >> > > > NOTICE: This message is intended for the use of the individual or >> >> > entity >> >> > > to >> >> > > > which it is addressed and may contain information that is >> >> confidential, >> >> > > > privileged and exempt from disclosure under applicable law. If >> the >> >> > reader >> >> > > > of this message is not the intended recipient, you are hereby >> >> notified >> >> > > that >> >> > > > any printing, copying, dissemination, distribution, disclosure or >> >> > > > forwarding of this communication is strictly prohibited. If you >> have >> >> > > > received this communication in error, please contact the sender >> >> > > immediately >> >> > > > and delete it from your system. Thank You. >> >> > > > >> >> > > >> >> > >> >> >> >> >> >> >> >> -- >> >> "...:::Aniket:::... Quetzalco@tl" >> >> >> >> -- >> CONFIDENTIALITY NOTICE >> NOTICE: This message is intended for the use of the individual or entity >> to >> which it is addressed and may contain information that is confidential, >> privileged and exempt from disclosure under applicable law. If the reader >> of this message is not the intended recipient, you are hereby notified >> that >> any printing, copying, dissemination, distribution, disclosure or >> forwarding of this communication is strictly prohibited. If you have >> received this communication in error, please contact the sender >> immediately >> and delete it from your system. Thank You. >> > > --001a11c3b956fb643a04fc0f371d--