hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jim Kellerman <...@powerset.com>
Subject RE: [VOTE] Release Hadoop 0.16.2 (candidate 0)
Date Mon, 31 Mar 2008 06:57:56 GMT
Yes, it does appear to be intermittent. hbase-0.1.0 was built against hadoop-0.16.0, so I assumed
that there were no incompatible changes. It just failed again with the exception:

2008-03-31 06:42:29,079 ERROR [regionserver/0:0:0:0:0:0:0:0:8020.compactor] hbase.HRegionServer$CompactSplitThread(267):
Compaction failed for region TestTable,0021017156,1206945717128
java.lang.NullPointerException
        at org.apache.hadoop.dfs.DFSClient$DFSOutputStream.close(DFSClient.java:2262)
        at org.apache.hadoop.fs.FSDataOutputStream$PositionCache.close(FSDataOutputStream.java:51)
        at org.apache.hadoop.fs.FSDataOutputStream.close(FSDataOutputStream.java:67)
        at org.apache.hadoop.hbase.HStoreFile.writeSplitInfo(HStoreFile.java:267)
        at org.apache.hadoop.hbase.HStoreFile.writeReferenceFiles(HStoreFile.java:246)
        at org.apache.hadoop.hbase.HStoreFile.splitStoreFile(HStoreFile.java:239)
        at org.apache.hadoop.hbase.HRegion.splitRegion(HRegion.java:740)
        at org.apache.hadoop.hbase.HRegionServer$CompactSplitThread.split(HRegionServer.java:291)
        at org.apache.hadoop.hbase.HRegionServer$CompactSplitThread.run(HRegionServer.java:255)

and if you look at the source for DFSClient for 0.16.0, line 2262 is:
2260:        synchronized (dataQueue) {
2261:          if (response != null) {
2262:            response.close();
2263:          }

Obviously the code in DFSClient$DFSOutputStream has changed considerably from 0.16.0 to 0.16.2.

My assumption was that applications built against 0.16.x should not have to be recompiled
for 0.16.y. Is that not correct?

---
Jim Kellerman, Senior Engineer; Powerset


> -----Original Message-----
> From: Nigel Daley [mailto:ndaley@yahoo-inc.com]
> Sent: Sunday, March 30, 2008 11:20 PM
> To: core-dev@hadoop.apache.org
> Subject: Re: [VOTE] Release Hadoop 0.16.2 (candidate 0)
>
> Perhaps it's an intermittent problem?  Can you run a couple
> more times?
>
> On Mar 30, 2008, at 11:12 PM, Jim Kellerman wrote:
>
> > I did not recompile hbase-0.1.0 and it ran fine with hadoop-0.16.
> > [01]. I made two runs against 0.16.2 and they both failed the same
> > way.
> >
> > However, I just re-ran against 0.16.2 and the test passed,
> so I must
> > have configured something incorrectly.
> > My bad!.
> >
> > +1 for 0.16.2
> >
> > ---
> > Jim Kellerman, Senior Engineer; Powerset
> >
> >
> >> -----Original Message-----
> >> From: Nigel Daley [mailto:ndaley@yahoo-inc.com]
> >> Sent: Sunday, March 30, 2008 10:31 PM
> >> To: core-dev@hadoop.apache.org
> >> Subject: Re: [VOTE] Release Hadoop 0.16.2 (candidate 0)
> >>
> >> Jim, did you recompile hbase with this new release?  The fact that
> >> the line numbers don't match means something is probably running
> >> using an old version.
> >>
> >> Nige
> >>
> >> On Mar 30, 2008, at 10:21 PM, Jim Kellerman wrote:
> >>
> >>> Yes, I am using candidate 0 which I downloaded from http://
> >>> people.apache.org/~nigel/hadoop-0.16.2-candidate-0/
> >>>
> >>> ---
> >>> Jim Kellerman, Senior Engineer; Powerset
> >>>
> >>>
> >>>> -----Original Message-----
> >>>> From: s29752-hadoopdev@yahoo.com
> >> [mailto:s29752-hadoopdev@yahoo.com]
> >>>> Sent: Sunday, March 30, 2008 6:30 PM
> >>>> To: core-dev@hadoop.apache.org
> >>>> Subject: Re: [VOTE] Release Hadoop 0.16.2 (candidate 0)
> >>>>
> >>>> Hi Jim,
> >>>>
> >>>> I checked the source codes.  There is no way to get
> exceptions in
> >>>> these two line in DFSClient.  Are you using candidate 0?
> >>>> 1506:     private long bytesCurBlock = 0; // bytes writen in
> >>>> current block
> >>>> 2262:          if (currentPacket != null) {
> >>>>
> >>>> Nicholas
> >>>>
> >>>> ----- Original Message ----
> >>>> From: Jim Kellerman <jim@powerset.com>
> >>>> To: "core-dev@hadoop.apache.org" <core-dev@hadoop.apache.org>
> >>>> Sent: Sunday, March 30, 2008 2:58:26 PM
> >>>> Subject: RE: [VOTE] Release Hadoop 0.16.2 (candidate 0)
> >>>>
> >>>> -1
> >>>>
> >>>> HBase-0.1.0 has a torture test we run to qualify releases.
> >>>> This test ran fine on Hadoop-0.16.[01] but failed on
> >>>> Hadoop-0.16.2 release candidate with the following exceptions:
> >>>>
> >>>> 2008-03-30 17:32:58,993 ERROR [IPC Server handler 5 on 8020]
> >>>> hbase.HLog(380): Could not append to log. Opening new log.
> >> Exception:
> >>>> java.io.IOException: Stream closed.
> >>>>         at
> >>>> org.apache.hadoop.dfs.DFSClient$DFSOutputStream.<init>(DFSClie
> >>>> nt.java:1506)
> >>>>         at
> >> org.apache.hadoop.dfs.DFSClient.create(DFSClient.java:382)
> >>>>         at
> >>>> org.apache.hadoop.dfs.DistributedFileSystem.create(Distributed
> >>>> FileSystem.java:123)
> >>>>         at
> org.apache.hadoop.fs.FileSystem.create(FileSystem.java:
> >>>> 436)
> >>>>         at
> >>>>
> org.apache.hadoop.io.SequenceFile$Writer.<init>(SequenceFile.java:
> >>>> 827)
> >>>>         at
> >>>>
> >>
> org.apache.hadoop.io.SequenceFile.createWriter(SequenceFile.java:379)
> >>>>         at
> >>>>
> >>
> org.apache.hadoop.io.SequenceFile.createWriter(SequenceFile.java:270)
> >>>>         at org.apache.hadoop.hbase.HLog.rollWriter(HLog.java:230)
> >>>>         at
> >>>> org.apache.hadoop.hbase.HRegionServer$LogRoller.run(HRegionSer
> >>>> ver.java:539)
> >>>> 2008-03-30 17:32:58,993 ERROR
> >>>> [regionserver/0:0:0:0:0:0:0:0:8020.logRoller]
> >>>> hbase.HRegionServer$LogRoller(545): Log rolling failed
> >>>> java.lang.NullPointerException
> >>>>         at
> >>>> org.apache.hadoop.dfs.DFSClient$DFSOutputStream.close(DFSClien
> >>>> t.java:2262)
> >>>>         at
> >>>> org.apache.hadoop.fs.FSDataOutputStream$PositionCache.close(FS
> >>>> DataOutputStream.java:51)
> >>>>         at
> >>>> org.apache.hadoop.fs.FSDataOutputStream.close(FSDataOutputStre
> >>>> am.java:67)
> >>>>         at
> >>>>
> >>
> org.apache.hadoop.io.SequenceFile$Writer.close(SequenceFile.java:932)
> >>>>         at org.apache.hadoop.hbase.HLog.rollWriter(HLog.java:217)
> >>>>         at
> >>>> org.apache.hadoop.hbase.HRegionServer$LogRoller.run(HRegionSer
> >>>> ver.java:539)
> >>>>
> >>>> ---
> >>>> Jim Kellerman, Senior Engineer; Powerset
> >>>>
> >>>>
> >>>>> -----Original Message-----
> >>>>> From: Nigel Daley [mailto:ndaley@yahoo-inc.com]
> >>>>> Sent: Friday, March 28, 2008 10:19 PM
> >>>>> To: core-dev@hadoop.apache.org
> >>>>> Subject: [VOTE] Release Hadoop 0.16.2 (candidate 0)
> >>>>>
> >>>>> I've created a candidate build for Hadoop 0.16.2.  This fixes
> >>>>> 25 issues in 0.16.1.  Now that HBase has it's own
> >> releases, it has
> >>>>> been removed from this release.
> >>>>>
> >>>>> Please download and try it out.
> >>>>>
> >>>>>    http://people.apache.org/~nigel/hadoop-0.16.2-candidate-0/
> >>>>>
> >>>>> Should we release this?  Vote closes on Wednesday, April 2.
> >>>>>
> >>>>> Nigel
> >>>>>
> >>>>>
> >>>>>
> >>>>> No virus found in this incoming message.
> >>>>> Checked by AVG.
> >>>>> Version: 7.5.519 / Virus Database: 269.22.1/1348 - Release
> >>>>> Date: 3/28/2008 10:58 AM
> >>>>>
> >>>>>
> >>>>
> >>>> No virus found in this outgoing message.
> >>>> Checked by AVG.
> >>>> Version: 7.5.519 / Virus Database: 269.22.1/1350 - Release
> >>>> Date: 3/30/2008 12:32 PM
> >>>>
> >>>>
> >>>>
> >>>>
> >>>>
> >>>> No virus found in this incoming message.
> >>>> Checked by AVG.
> >>>> Version: 7.5.519 / Virus Database: 269.22.1/1350 - Release
> >>>> Date: 3/30/2008 12:32 PM
> >>>>
> >>>>
> >>>
> >>> No virus found in this outgoing message.
> >>> Checked by AVG.
> >>> Version: 7.5.519 / Virus Database: 269.22.1/1350 - Release Date:
> >>> 3/30/2008 12:32 PM
> >>>
> >>
> >>
> >> No virus found in this incoming message.
> >> Checked by AVG.
> >> Version: 7.5.519 / Virus Database: 269.22.1/1350 - Release
> >> Date: 3/30/2008 12:32 PM
> >>
> >>
> >
> > No virus found in this outgoing message.
> > Checked by AVG.
> > Version: 7.5.519 / Virus Database: 269.22.1/1350 - Release Date:
> > 3/30/2008 12:32 PM
> >
>
>
> No virus found in this incoming message.
> Checked by AVG.
> Version: 7.5.519 / Virus Database: 269.22.1/1350 - Release
> Date: 3/30/2008 12:32 PM
>
>

No virus found in this outgoing message.
Checked by AVG.
Version: 7.5.519 / Virus Database: 269.22.1/1350 - Release Date: 3/30/2008 12:32 PM


Mime
View raw message