hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Mark Vigeant <mark.vige...@riskmetrics.com>
Subject RE: hbase-test
Date Mon, 30 Nov 2009 20:59:05 GMT
Ok, thank you.

I'm running the test on my master machine right now without hbase or hadoop running, and already
I'm seeing the same tests failing...

Additionally I don't know how to check the classpath for a different version of hbase... I
haven't installed any other versions though so I believe it's impossible for there to be 2
conflicting versions.


-----Original Message-----
From: jdcryans@gmail.com [mailto:jdcryans@gmail.com] On Behalf Of Jean-Daniel Cryans
Sent: Monday, November 30, 2009 2:49 PM
To: hbase-user@hadoop.apache.org
Subject: Re: hbase-test

Name of thread is "Region server throwing NPE"

Well usually it doesn't use the hbase-site.xml file and even more than
that, it doesn't use a hadoop setup at all. Try shutting down Hadoop
and HBase on that node before running the tests.

J-D

On Mon, Nov 30, 2009 at 11:45 AM, Mark Vigeant
<mark.vigeant@riskmetrics.com> wrote:
> Also, this is the first time I'm running hbase on these machines. I was running hbase
0.20.1 on other machines, but this is the only version of hbase I've got on all 4 of my machines
(0.20.2)
>
> -----Original Message-----
> From: Mark Vigeant [mailto:mark.vigeant@riskmetrics.com]
> Sent: Monday, November 30, 2009 2:39 PM
> To: hbase-user@hadoop.apache.org
> Subject: RE: hbase-test
>
> Do you remember the name of the thread? I'll try searching for it.
>
> And yes, I installed HBase and created a test table or 2 before running the tests. I
have 1 master and 3 region servers. How do I make it not use the hbase-site.xml?
>
> -----Original Message-----
> From: jdcryans@gmail.com [mailto:jdcryans@gmail.com] On Behalf Of Jean-Daniel Cryans
> Sent: Monday, November 30, 2009 2:26 PM
> To: hbase-user@hadoop.apache.org
> Subject: Re: hbase-test
>
> This error seems like a version mismatch. There was a thread recently
> about it on this mailing list.
>
> WRT to the errors, it seems the tests in your case are using your
> hbase-site.xml file and it should not be. Also is HBase already
> running on that machine?
>
> J-D
>
> On Mon, Nov 30, 2009 at 9:16 AM, Mark Vigeant
> <mark.vigeant@riskmetrics.com> wrote:
>> I would also like to add that when I try to upload information to a table through
a java program I wrote (that worked on 0.20.1) the table gets created, but no data is inserted
into it. The following error is displayed:
>>
>> Exception in thread "main" java.lang.reflect.UndeclaredThrowableException
>>        at $Proxy1.getRegionInfo(Unknown Source)
>>        at org.apache.hadoop.hbase.client.HConnectionManager$TableServers.locateRootRegion(HConnectionManager.java:931)
>>        at org.apache.hadoop.hbase.client.HConnectionManager$TableServers.locateRegion(HConnectionManager.java:573)
>>        at org.apache.hadoop.hbase.client.HConnectionManager$TableServers.locateRegion(HConnectionManager.java:549)
>>        at org.apache.hadoop.hbase.client.HConnectionManager$TableServers.locateRegionInMeta(HConnectionManager.java:623)
>>        at org.apache.hadoop.hbase.client.HConnectionManager$TableServers.locateRegion(HConnectionManager.java:582)
>>        at org.apache.hadoop.hbase.client.HConnectionManager$TableServers.locateRegion(HConnectionManager.java:549)
>>        at org.apache.hadoop.hbase.client.HConnectionManager$TableServers.locateRegionInMeta(HConnectionManager.java:623)
>>        at org.apache.hadoop.hbase.client.HConnectionManager$TableServers.locateRegion(HConnectionManager.java:586)
>>        at org.apache.hadoop.hbase.client.HConnectionManager$TableServers.locateRegion(HConnectionManager.java:549)
>>        at org.apache.hadoop.hbase.client.HBaseAdmin.createTable(HBaseAdmin.java:169)
>>        at com.riskmetrics.hadoop.loganalysis.hbase.uploader.TableUploader.main(TableUploader.java:256)
>> Caused by: org.apache.hadoop.ipc.RemoteException: java.io.IOException: java.lang.NullPointerException
>>        at java.lang.Class.searchMethods(Class.java:2646)
>>        at java.lang.Class.getMethod0(Class.java:2670)
>>        at java.lang.Class.getMethod(Class.java:1603)
>>        at org.apache.hadoop.hbase.ipc.HBaseRPC$Server.call(HBaseRPC.java:643)
>>        at org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:915)
>>
>>        at org.apache.hadoop.hbase.ipc.HBaseClient.call(HBaseClient.java:723)
>>        at org.apache.hadoop.hbase.ipc.HBaseRPC$Invoker.invoke(HBaseRPC.java:328)
>>        ... 12 more
>>
>> Thank you for all your help!
>>
>> -Mark
>>
>> -----Original Message-----
>> From: Mark Vigeant [mailto:mark.vigeant@riskmetrics.com]
>> Sent: Monday, November 30, 2009 11:14 AM
>> To: hbase-user@hadoop.apache.org
>> Subject: RE: hbase-test
>>
>> Hey J-D
>>
>> That is the error that shows up at the end of most failed TEST-* files. 2 interesting
things to note: none of the tests had "failures," only "errors." This still caused them to
fail. Also, the tests each ran *mini HDFS clusters* instead of running on top of my existing
one. Is that supposed to happen? The list of failed tests is huge.
>>
>> Here is the complete list of failed tests:
>> TestBatchUpdate, TestClient, TestForceSplit, TestGet, TestGetRowVersions, TestHBaseAdmin,
TestHTable, TestListTables, TestOldAPIGetRowVersions, TestOldAPIHTable, TestOldAPITimestamp,
TestPut, TestScannerTimes, TestTimestamp, TestTableIndex, TestTableMapReduce, TestTableIndex,
TestTableMapReduce, TestTimeRangeMapRed, TestRegionManager, TestCompaction, TestGetClosestAtOrBefore,
TestHLog, TestLogRolling, TestScanner, TestStoreFile, TestEmptyMetaInfo, TestHBaseCluster,
TestInfoServers, TestMasterAdmin, TestMergeMeta, TestMergeTable, TestRegionRebalancing, TestScanMultipleVersions,
TestTable, TestZooKeeper, TestThriftServer,and TestMergeTool
>>
>> The same error was caused by the following four problems. I suspect there's some
weird lack of communication between my namenode and my hbase cluster?
>>
>> Could not get block locations. Source file "/user/hadoop/.META./1028785192/.regioninfo"
>> Could not get block locations. Source file "/hbase/testSplit/hlog.dat.1259154481471"
>> Could not get block locations. Source file "/user/hadoop/hbase.version"
>> Could not get block locations. Source file "/user/hadoop/regionname/familyname/7117946442331006053"
>>
>> TestStopRow had these different exceptions:
>> java.lang.NullPointerException
>>        at org.apache.hadoop.hbase.regionserver.TestScanner.testStopRow(TestScanner.java:142)
>> java.io.IOException: Cannot lock storage /home/hadoop/hbase-0.20.2/build/test/data/dfs/name1.
The directory is already locked.
>>
>>
>> Another interesting thing to point out is that in all the failed tests there are
a few warnings that come up a lot (if not every time):
>> 2009-11-25 12:12:01,539 WARN  [Thread-60] hdfs.DFSClient$DFSOutputStream$DataStreamer(2304):
DataStreamer Exception:    java.lang.NumberFormatException: For input string: "0:0:0:0:0:0:1:36361"
>> 2009-11-25 11:36:42,163 WARN  [org.apache.hadoop.hdfs.server.datanode.DataXceiverServer@63a5ec6c]
datanode.DataXceiverServer(137):      DatanodeRegistration(0:0:0:0:0:0:0:1:42220, storageID=DS-325721370-10.7.20.54-42220-1259152601715,
infoPort=56993,      ipcPort=47506):DataXceiveServer: java.nio.channels.AsynchronousCloseException
>>
>> Thank you so much for your time, I apologize for the long email. I really want to
get this to work, and I have a feeling that so many errors in the tests will prevent me from
getting it to. Thanks!
>>
>> -Mark
>>
>>
>> -----Original Message-----
>> From: jdcryans@gmail.com [mailto:jdcryans@gmail.com] On Behalf Of Jean-Daniel Cryans
>> Sent: Wednesday, November 25, 2009 7:04 PM
>> To: hbase-user@hadoop.apache.org
>> Subject: Re: hbase-test
>>
>> That error means the Namenode isn't giving the locations for a file in
>> the namespace, I don't think this is the error you are searching for.
>>
>> Which tests failed? Which version? Is it because of errors or because
>> of failed assertions? Look at the end of the TEST-* files to figure
>> it.
>>
>> J-D
>>
>> On Wed, Nov 25, 2009 at 8:37 AM, Mark Vigeant
>> <mark.vigeant@riskmetrics.com> wrote:
>>> I just ran the "ant test" and a bunch of the tests failed. Upon looking at the
logs, this one error came up:
>>>
>>> Could not get block locations. Source file "/user/hadoop/hbase.version" - Aborting...
>>> java.io.IOException: Could not get block locations. Source file "/user/hadoop/hbase.version"
- Aborting...
>>>        at org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.processDatanodeError(DFSClient.java:2505)
>>>        at org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.access$1600(DFSClient.java:2078)
>>>        at org.apache.hadoop.hdfs.DFSClient$DFSOutputStream$DataStreamer.run(DFSClient.java:2241)
>>>
>>> I found the file hbase.version somewhere else on the dfs and I moved it to /user/hadoop/
and ran it again but the tests are still failing for the same reason. Any thoughts?
>>>
>>> -Mark
>>> -----Original Message-----
>>> From: Ryan Rawson [mailto:ryanobjc@gmail.com]
>>> Sent: Tuesday, November 24, 2009 6:04 PM
>>> To: hbase-user@hadoop.apache.org
>>> Subject: Re: hbase-test
>>>
>>> A follow up - our jars are also not standalone, they have a number of
>>> dependencies which live in lib/*.  The shell scripts in bin/* can boot
>>> up and run hbase and other tools, and set the classpath automatically.
>>>  If you dont want to do that for some reason, add every jar in lib/*
>>> and the built jars in 'build'.
>>>
>>> -ryan
>>>
>>> On Tue, Nov 24, 2009 at 3:01 PM, Mark Vigeant
>>> <mark.vigeant@riskmetrics.com> wrote:
>>>> To be more clear the error I get when I run
>>>> Java -jar hbase-0.20.2-test.jar
>>>> Is
>>>> Exception in thread "main" java.lang.NoClassDefFoundError: org/apache/hadoop/conf/Configuration
>>>> Caused by:
>>>>
>>>> ....
>>>>
>>>> Could not find the main class: org/apache/hadoop/hbase/PerformanceEvaluation
>>>>
>>>> -----Original Message-----
>>>> From: Mark Vigeant [mailto:mark.vigeant@riskmetrics.com]
>>>> Sent: Tuesday, November 24, 2009 5:59 PM
>>>> To: hbase-user@hadoop.apache.org
>>>> Subject: hbase-test
>>>>
>>>> Hey-
>>>>
>>>> Quick question: what are the classes / how do I run the hbase-test .jar in
0.20.2? I'm getting the error that there's no main class and I can't find anything written
about it.
>>>>
>>>> Thanks!
>>>>
>>>> Mark Vigeant
>>>> RiskMetrics Group, Inc.
>>>> One Chase Manhattan Plaza
>>>> 44th Floor
>>>> New York, NY 10005
>>>> (p) 646-778-4142
>>>>
>>>>
>>>> This email message and any attachments are for the sole use of the intended
recipients and may contain proprietary and/or confidential information which may be privileged
or otherwise protected from disclosure. Any unauthorized review, use, disclosure or distribution
is prohibited. If you are not an intended recipient, please contact the sender by reply email
and destroy the original message and any copies of the message as well as any attachments
to the original message.
>>>>
>>>> This email message and any attachments are for the sole use of the intended
recipients and may contain proprietary and/or confidential information which may be privileged
or otherwise protected from disclosure. Any unauthorized review, use, disclosure or distribution
is prohibited. If you are not an intended recipient, please contact the sender by reply email
and destroy the original message and any copies of the message as well as any attachments
to the original message.
>>>>
>>>
>>> This email message and any attachments are for the sole use of the intended recipients
and may contain proprietary and/or confidential information which may be privileged or otherwise
protected from disclosure. Any unauthorized review, use, disclosure or distribution is prohibited.
If you are not an intended recipient, please contact the sender by reply email and destroy
the original message and any copies of the message as well as any attachments to the original
message.
>>>
>>
>> This email message and any attachments are for the sole use of the intended recipients
and may contain proprietary and/or confidential information which may be privileged or otherwise
protected from disclosure. Any unauthorized review, use, disclosure or distribution is prohibited.
If you are not an intended recipient, please contact the sender by reply email and destroy
the original message and any copies of the message as well as any attachments to the original
message.
>>
>> This email message and any attachments are for the sole use of the intended recipients
and may contain proprietary and/or confidential information which may be privileged or otherwise
protected from disclosure. Any unauthorized review, use, disclosure or distribution is prohibited.
If you are not an intended recipient, please contact the sender by reply email and destroy
the original message and any copies of the message as well as any attachments to the original
message.
>>
>
> This email message and any attachments are for the sole use of the intended recipients
and may contain proprietary and/or confidential information which may be privileged or otherwise
protected from disclosure. Any unauthorized review, use, disclosure or distribution is prohibited.
If you are not an intended recipient, please contact the sender by reply email and destroy
the original message and any copies of the message as well as any attachments to the original
message.
>
> This email message and any attachments are for the sole use of the intended recipients
and may contain proprietary and/or confidential information which may be privileged or otherwise
protected from disclosure. Any unauthorized review, use, disclosure or distribution is prohibited.
If you are not an intended recipient, please contact the sender by reply email and destroy
the original message and any copies of the message as well as any attachments to the original
message.
>

This email message and any attachments are for the sole use of the intended recipients and
may contain proprietary and/or confidential information which may be privileged or otherwise
protected from disclosure. Any unauthorized review, use, disclosure or distribution is prohibited.
If you are not an intended recipient, please contact the sender by reply email and destroy
the original message and any copies of the message as well as any attachments to the original
message.

Mime
View raw message