hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Schlesiger, Chris (Civ, ARL/CISD)" <chris.schlesi...@us.army.mil>
Subject Custom WritableByteArrayComparable crashing. Need help please (UNCLASSIFIED)
Date Fri, 11 Jun 2010 17:12:36 GMT
Classification:  UNCLASSIFIED 
Caveats: NONE

Our HBase contains people names for RowIDs, and we need our application
to be able to try other types of string matching to find rows other than
than the straight Substring and Regex matching that comes with the HBase
API in the filters package.

We were first working with HBase 0.20.1 where
WritableByteArrayComparable was an interface. I tried implementing the
interface and had no luck. I figured I'd better upgrade to the newest
HBase, which I just did. Now in 0.20.4, WritableByteArrayComparable is
an abstract class. Okay, so I changed my code to create a new child
class and I still ran into the same problem with crashing and no rows
returned by the scanner.

So I decided to just try out the mere idea of creating a new child
comparator to see if the same problem arose. I went to the HBase source
code for the 0.20.4 tag, and looked at the SubstringComparator class in
the filters package. I re-created it line by line in my own java
project. The class I created was the same except that since I wasn't in
the same package as the super class, I couldn't access the .value
property as on line 74. But it looked to me that it wasn't needed since
the getValue() method was overridden and ignoring .value anyway.

I created a scanner, created a RowFilter and passed in my
SubstringComparatorDuplicate, and ran the program. Same exact crash. If
I change my code only to create a SubstringComparator from the hbase jar
instead, it works perfectly and I get rows returned by the scanner.

Here are my symptoms:

The program halts for almost 30 seconds, then I get the following
exception:

org.apache.hadoop.hbase.client.RetriesExhaustedException: Trying to
contact region server 127.0.0.1:60062 for region People,,1276369002801,
row '', but failed after 10 attempts.
Exceptions:
java.io.IOException: Call to /127.0.0.1:60062 failed on local exception:
java.io.EOFException
java.io.IOException: Call to /127.0.0.1:60062 failed on local exception:
java.io.EOFException
java.io.IOException: Call to /127.0.0.1:60062 failed on local exception:
java.io.EOFException
java.io.IOException: Call to /127.0.0.1:60062 failed on local exception:
java.io.EOFException
java.io.IOException: Call to /127.0.0.1:60062 failed on local exception:
java.io.EOFException
java.io.IOException: Call to /127.0.0.1:60062 failed on local exception:
java.io.EOFException
java.io.IOException: Call to /127.0.0.1:60062 failed on local exception:
java.io.EOFException
java.io.IOException: Call to /127.0.0.1:60062 failed on local exception:
java.io.EOFException
java.io.IOException: Call to /127.0.0.1:60062 failed on local exception:
java.io.EOFException
java.io.IOException: Call to /127.0.0.1:60062 failed on local exception:
java.io.EOFException

	at
org.apache.hadoop.hbase.client.HConnectionManager$TableServers.getRegion
ServerWithRetries(HConnectionManager.java:1055)
	at
org.apache.hadoop.hbase.client.Htable$ClientScanner.nextScanner(HTable.j
ava:2003)
	at
org.apache.hadoop.hbase.client.Htable$ClientScanner.initialize(HTable.ja
va:1923)
	at
org.apache.hadoop.hbase.client.Htable.getScanner(HTable.java:403)
	at org.mycompany.test.search(Search.java:209)
	at org.mycompany.test.main(Main.java:40)

I tried putting break points on each of the methods in my
SubstringComparatorDuplicate, and the breakpoint in the write method
gets hit 10 times and then I get the exception.

I'm running HBase in pseudo-distributed mode on my development computer
until I get the code working. The database is small. I'm running Hadoop
0.20.2 in dfs mode only for the same reason.

I can't see any reason why my SubstringComparatorDuplicate class
shouldn't work. It is identical to the code found here

http://svn.apache.org/viewvc/hbase/tags/0.20.4/src/java/org/apache/hadoo
p/hbase/filter/SubstringComparator.java?view=markup

With only 2 exceptions.
1) I'm in a different package, project, and jar file.
2) Line 74 is commented out

Has anyone got any great ideas? I'm pretty much stuck now, and it's
looking like there's a bug in the HBase code as near as I can tell.

I'd appreciate the help. Thanks in advance.

Chris Schlesiger
cschles@arl.army.mil
Electronics/Software Engineer
U.S. Army Research Lab
Voc: 301-394-2473
Fax: 301-394-3591
DSN 290-2473
Classification:  UNCLASSIFIED 
Caveats: NONE


Mime
View raw message