Mailing-List: contact core-dev-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: core-dev@hadoop.apache.org
Received-SPF: pass (nike.apache.org: domain of jim@powerset.com designates
 207.5.72.93 as permitted sender)
From: Jim Kellerman <jim@powerset.com>
To: "core-dev@hadoop.apache.org" <core-dev@hadoop.apache.org>
Date: Mon, 31 Mar 2008 02:07:19 -0700
Subject: RE: [VOTE] Release Hadoop 0.16.2 (candidate 0)
Thread-Topic: [VOTE] Release Hadoop 0.16.2 (candidate 0)
Thread-Index: AciS94OsCvJJi26ESbK+vkxZELqQwgAA5tzwAADpyrA=
Message-ID: 
 <84E2AE771361E9419DD0EFBD31F09C4D4F59E12235@EXVMBX015-1.exch015.msoutlookonline.net>
References: <685408.65105.qm@web56205.mail.re3.yahoo.com>
 <84E2AE771361E9419DD0EFBD31F09C4D4F59E12230@EXVMBX015-1.exch015.msoutlookonline.net>
 <0A9E81FD-1F51-4CF7-A7FE-2138A48CC242@yahoo-inc.com>
 <84E2AE771361E9419DD0EFBD31F09C4D4F59E12232@EXVMBX015-1.exch015.msoutlookonline.net>
 <8BA1151E-3280-424E-A106-5870EDFF670E@yahoo-inc.com>
 <84E2AE771361E9419DD0EFBD31F09C4D4F59E12234@EXVMBX015-1.exch015.msoutlookonline.net>
In-Reply-To: 
 <84E2AE771361E9419DD0EFBD31F09C4D4F59E12234@EXVMBX015-1.exch015.msoutlookonline.net>
Accept-Language: en-US
Content-Language: en-US
acceptlanguage: en-US
Content-Type: text/plain; charset="windows-1250"
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0

Assuming that applications built against 0.16.x should not have to be recom=
piled for 0.16.x+1, I have to assume that an intermittent bug has been intr=
oduced between hadoop 0.16.1 and 0.16.2 because I have run the HBase tortur=
e test 6 times against both 0.16.1 and 0.16.2. Hadoop 0.16.2 failed 3 out o=
f 6 times.
Hadoop 0.16.1 succeeded 6 out of 6 times.

If a recompile between 0.16.1 and 0.16.2 is required, then 0.16.2 gets a -1=
.

If no recompile should be required, then 0.16.2 gets a -1 because it fails
50% of the time where 0.16.1 succeeded.

---
Jim Kellerman, Senior Engineer; Powerset


> -----Original Message-----
> From: Jim Kellerman [mailto:jim@powerset.com]
> Sent: Sunday, March 30, 2008 11:58 PM
> To: core-dev@hadoop.apache.org
> Subject: RE: [VOTE] Release Hadoop 0.16.2 (candidate 0)
>
> Yes, it does appear to be intermittent. hbase-0.1.0 was built
> against hadoop-0.16.0, so I assumed that there were no
> incompatible changes. It just failed again with the exception:
>
> 2008-03-31 06:42:29,079 ERROR
> [regionserver/0:0:0:0:0:0:0:0:8020.compactor]
> hbase.HRegionServer$CompactSplitThread(267): Compaction
> failed for region TestTable,0021017156,1206945717128
> java.lang.NullPointerException
>         at
> org.apache.hadoop.dfs.DFSClient$DFSOutputStream.close(DFSClien
> t.java:2262)
>         at
> org.apache.hadoop.fs.FSDataOutputStream$PositionCache.close(FS
> DataOutputStream.java:51)
>         at
> org.apache.hadoop.fs.FSDataOutputStream.close(FSDataOutputStre
> am.java:67)
>         at
> org.apache.hadoop.hbase.HStoreFile.writeSplitInfo(HStoreFile.java:267)
>         at
> org.apache.hadoop.hbase.HStoreFile.writeReferenceFiles(HStoreF
> ile.java:246)
>         at
> org.apache.hadoop.hbase.HStoreFile.splitStoreFile(HStoreFile.java:239)
>         at
> org.apache.hadoop.hbase.HRegion.splitRegion(HRegion.java:740)
>         at
> org.apache.hadoop.hbase.HRegionServer$CompactSplitThread.split
> (HRegionServer.java:291)
>         at
> org.apache.hadoop.hbase.HRegionServer$CompactSplitThread.run(H
> RegionServer.java:255)
>
> and if you look at the source for DFSClient for 0.16.0, line 2262 is:
> 2260:        synchronized (dataQueue) {
> 2261:          if (response !=3D null) {
> 2262:            response.close();
> 2263:          }
>
> Obviously the code in DFSClient$DFSOutputStream has changed
> considerably from 0.16.0 to 0.16.2.
>
> My assumption was that applications built against 0.16.x
> should not have to be recompiled for 0.16.y. Is that not correct?
>
> ---
> Jim Kellerman, Senior Engineer; Powerset
>
>
> > -----Original Message-----
> > From: Nigel Daley [mailto:ndaley@yahoo-inc.com]
> > Sent: Sunday, March 30, 2008 11:20 PM
> > To: core-dev@hadoop.apache.org
> > Subject: Re: [VOTE] Release Hadoop 0.16.2 (candidate 0)
> >
> > Perhaps it's an intermittent problem?  Can you run a couple more
> > times?
> >
> > On Mar 30, 2008, at 11:12 PM, Jim Kellerman wrote:
> >
> > > I did not recompile hbase-0.1.0 and it ran fine with hadoop-0.16.
> > > [01]. I made two runs against 0.16.2 and they both failed
> the same
> > > way.
> > >
> > > However, I just re-ran against 0.16.2 and the test passed,
> > so I must
> > > have configured something incorrectly.
> > > My bad!.
> > >
> > > +1 for 0.16.2
> > >
> > > ---
> > > Jim Kellerman, Senior Engineer; Powerset
> > >
> > >
> > >> -----Original Message-----
> > >> From: Nigel Daley [mailto:ndaley@yahoo-inc.com]
> > >> Sent: Sunday, March 30, 2008 10:31 PM
> > >> To: core-dev@hadoop.apache.org
> > >> Subject: Re: [VOTE] Release Hadoop 0.16.2 (candidate 0)
> > >>
> > >> Jim, did you recompile hbase with this new release?  The
> fact that
> > >> the line numbers don't match means something is probably running
> > >> using an old version.
> > >>
> > >> Nige
> > >>
> > >> On Mar 30, 2008, at 10:21 PM, Jim Kellerman wrote:
> > >>
> > >>> Yes, I am using candidate 0 which I downloaded from http://
> > >>> people.apache.org/~nigel/hadoop-0.16.2-candidate-0/
> > >>>
> > >>> ---
> > >>> Jim Kellerman, Senior Engineer; Powerset
> > >>>
> > >>>
> > >>>> -----Original Message-----
> > >>>> From: s29752-hadoopdev@yahoo.com
> > >> [mailto:s29752-hadoopdev@yahoo.com]
> > >>>> Sent: Sunday, March 30, 2008 6:30 PM
> > >>>> To: core-dev@hadoop.apache.org
> > >>>> Subject: Re: [VOTE] Release Hadoop 0.16.2 (candidate 0)
> > >>>>
> > >>>> Hi Jim,
> > >>>>
> > >>>> I checked the source codes.  There is no way to get
> > exceptions in
> > >>>> these two line in DFSClient.  Are you using candidate 0?
> > >>>> 1506:     private long bytesCurBlock =3D 0; // bytes writen in
> > >>>> current block
> > >>>> 2262:          if (currentPacket !=3D null) {
> > >>>>
> > >>>> Nicholas
> > >>>>
> > >>>> ----- Original Message ----
> > >>>> From: Jim Kellerman <jim@powerset.com>
> > >>>> To: "core-dev@hadoop.apache.org" <core-dev@hadoop.apache.org>
> > >>>> Sent: Sunday, March 30, 2008 2:58:26 PM
> > >>>> Subject: RE: [VOTE] Release Hadoop 0.16.2 (candidate 0)
> > >>>>
> > >>>> -1
> > >>>>
> > >>>> HBase-0.1.0 has a torture test we run to qualify releases.
> > >>>> This test ran fine on Hadoop-0.16.[01] but failed on
> > >>>> Hadoop-0.16.2 release candidate with the following exceptions:
> > >>>>
> > >>>> 2008-03-30 17:32:58,993 ERROR [IPC Server handler 5 on 8020]
> > >>>> hbase.HLog(380): Could not append to log. Opening new log.
> > >> Exception:
> > >>>> java.io.IOException: Stream closed.
> > >>>>         at
> > >>>> org.apache.hadoop.dfs.DFSClient$DFSOutputStream.<init>(DFSClie
> > >>>> nt.java:1506)
> > >>>>         at
> > >> org.apache.hadoop.dfs.DFSClient.create(DFSClient.java:382)
> > >>>>         at
> > >>>> org.apache.hadoop.dfs.DistributedFileSystem.create(Distributed
> > >>>> FileSystem.java:123)
> > >>>>         at
> > org.apache.hadoop.fs.FileSystem.create(FileSystem.java:
> > >>>> 436)
> > >>>>         at
> > >>>>
> > org.apache.hadoop.io.SequenceFile$Writer.<init>(SequenceFile.java:
> > >>>> 827)
> > >>>>         at
> > >>>>
> > >>
> >
> org.apache.hadoop.io.SequenceFile.createWriter(SequenceFile.java:379)
> > >>>>         at
> > >>>>
> > >>
> >
> org.apache.hadoop.io.SequenceFile.createWriter(SequenceFile.java:270)
> > >>>>         at
> org.apache.hadoop.hbase.HLog.rollWriter(HLog.java:230)
> > >>>>         at
> > >>>> org.apache.hadoop.hbase.HRegionServer$LogRoller.run(HRegionSer
> > >>>> ver.java:539)
> > >>>> 2008-03-30 17:32:58,993 ERROR
> > >>>> [regionserver/0:0:0:0:0:0:0:0:8020.logRoller]
> > >>>> hbase.HRegionServer$LogRoller(545): Log rolling failed
> > >>>> java.lang.NullPointerException
> > >>>>         at
> > >>>> org.apache.hadoop.dfs.DFSClient$DFSOutputStream.close(DFSClien
> > >>>> t.java:2262)
> > >>>>         at
> > >>>> org.apache.hadoop.fs.FSDataOutputStream$PositionCache.close(FS
> > >>>> DataOutputStream.java:51)
> > >>>>         at
> > >>>> org.apache.hadoop.fs.FSDataOutputStream.close(FSDataOutputStre
> > >>>> am.java:67)
> > >>>>         at
> > >>>>
> > >>
> >
> org.apache.hadoop.io.SequenceFile$Writer.close(SequenceFile.java:932)
> > >>>>         at
> org.apache.hadoop.hbase.HLog.rollWriter(HLog.java:217)
> > >>>>         at
> > >>>> org.apache.hadoop.hbase.HRegionServer$LogRoller.run(HRegionSer
> > >>>> ver.java:539)
> > >>>>
> > >>>> ---
> > >>>> Jim Kellerman, Senior Engineer; Powerset
> > >>>>
> > >>>>
> > >>>>> -----Original Message-----
> > >>>>> From: Nigel Daley [mailto:ndaley@yahoo-inc.com]
> > >>>>> Sent: Friday, March 28, 2008 10:19 PM
> > >>>>> To: core-dev@hadoop.apache.org
> > >>>>> Subject: [VOTE] Release Hadoop 0.16.2 (candidate 0)
> > >>>>>
> > >>>>> I've created a candidate build for Hadoop 0.16.2.  This fixes
> > >>>>> 25 issues in 0.16.1.  Now that HBase has it's own
> > >> releases, it has
> > >>>>> been removed from this release.
> > >>>>>
> > >>>>> Please download and try it out.
> > >>>>>
> > >>>>>    http://people.apache.org/~nigel/hadoop-0.16.2-candidate-0/
> > >>>>>
> > >>>>> Should we release this?  Vote closes on Wednesday, April 2.
> > >>>>>
> > >>>>> Nigel
> > >>>>>
> > >>>>>
> > >>>>>
> > >>>>> No virus found in this incoming message.
> > >>>>> Checked by AVG.
> > >>>>> Version: 7.5.519 / Virus Database: 269.22.1/1348 - Release
> > >>>>> Date: 3/28/2008 10:58 AM
> > >>>>>
> > >>>>>
> > >>>>
> > >>>> No virus found in this outgoing message.
> > >>>> Checked by AVG.
> > >>>> Version: 7.5.519 / Virus Database: 269.22.1/1350 - Release
> > >>>> Date: 3/30/2008 12:32 PM
> > >>>>
> > >>>>
> > >>>>
> > >>>>
> > >>>>
> > >>>> No virus found in this incoming message.
> > >>>> Checked by AVG.
> > >>>> Version: 7.5.519 / Virus Database: 269.22.1/1350 - Release
> > >>>> Date: 3/30/2008 12:32 PM
> > >>>>
> > >>>>
> > >>>
> > >>> No virus found in this outgoing message.
> > >>> Checked by AVG.
> > >>> Version: 7.5.519 / Virus Database: 269.22.1/1350 - Release Date:
> > >>> 3/30/2008 12:32 PM
> > >>>
> > >>
> > >>
> > >> No virus found in this incoming message.
> > >> Checked by AVG.
> > >> Version: 7.5.519 / Virus Database: 269.22.1/1350 - Release
> > >> Date: 3/30/2008 12:32 PM
> > >>
> > >>
> > >
> > > No virus found in this outgoing message.
> > > Checked by AVG.
> > > Version: 7.5.519 / Virus Database: 269.22.1/1350 - Release Date:
> > > 3/30/2008 12:32 PM
> > >
> >
> >
> > No virus found in this incoming message.
> > Checked by AVG.
> > Version: 7.5.519 / Virus Database: 269.22.1/1350 - Release
> > Date: 3/30/2008 12:32 PM
> >
> >
>
> No virus found in this outgoing message.
> Checked by AVG.
> Version: 7.5.519 / Virus Database: 269.22.1/1350 - Release
> Date: 3/30/2008 12:32 PM
>
>
> No virus found in this incoming message.
> Checked by AVG.
> Version: 7.5.519 / Virus Database: 269.22.1/1350 - Release
> Date: 3/30/2008 12:32 PM
>
>

No virus found in this outgoing message.
Checked by AVG.
Version: 7.5.519 / Virus Database: 269.22.1/1350 - Release Date: 3/30/2008 =
12:32 PM